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ABSTRACT 


Twenty four pilots flew simulated missions in an unmanned air vehicle (UAV) simulator 
under both single and dual UAV control, and in three conditions: a baseline condition, a 
condition in which certain information was displayed auditorally, to offload the heavy visual 
demands of the mission, and a condition in which flight path tracking was automated. Three 
tasks were perfonned within each UAV workstation: (1) Meeting the mission goals, by flying to 
10 command target waypoints and reporting intelligence information at each of these command 
targets, (2) monitoring a 3D image display for targets of opportunity on the ground below the 
flight path,(3) monitoring the health of on-board system parameters. Upon reaching a command 
target, or seeing a target of opportunity, pilots were required to enter a loiter pattern, zoom in and 
inspect the image. Pilots could also retrieve command target coordinates and report information 
at any time they wished. The data were evaluated in the context of three models of concurrent 
task performance, strict single channel theory, single resource theory and multiple resource 
theory. 


The results indicated a cost to dual UAV control in all three tasks, although this cost 
varied in its magnitude. The results also indicated that both the auditory and the automation 
assistance improved performance, and reduced the dual task decrement, relative to the baseline 
condition. In particular, the auditory display of system parameter failures enabled a large degree 
of parallel processing. Various analyses were carried out to examine the extent to which models 
based on each of the three attention theories were adequate in predicting the data. Some aspects 
of the data were consistent with each model. Thus a valid model to account for all aspects of the 
task would need to incorporate mechanisms based on each model. A separate section of the 
results applies the Army’s IMPRINT model to predicting the workload imposed by the various 
conditions. 
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1.0 Introduction 


Since the first use of airplanes in war over Tripoli, Libya in 1911 (Milnet, 2001), 
hundreds of thousands of pilots have been killed or sustained career-ending injuries. Most of 
these accidents occur during combat situations, but many can also be ascribed to training 
accidents and peacetime flying. The Air Force, Navy and Anny all support air combat units and 
spend millions of dollars each year training and preparing pilots to take on a multitude of combat 
roles (e.g., fighters, bombers, helicopters, transports, etc.). When one of their pilots is killed or 
injured, the financial loss to the military is significant, not to mention the emotional loss to their 
units and families. It has always been a priority of the military to find ways to protect their pilots 
and keep them from danger while still fulfilling the mission requirements. 

Within the past few decades, the US military has made a concerted effort to produce 
Unmanned Aerial Vehicles (UAVs) to fulfill many mission requirements without exposing a 
human pilot to combat danger. A UAV can be flown by a specially trained pilot from a remote 
location hundreds or thousands of miles from the actual combat situation. These aircraft are 
cheaper to produce than nonnal warplanes, and when one of them gets shot down, the financial 
loss is significantly smaller. These aircraft can also carry out missions that would expose human 
pilots to extreme combat or environmental dangers that were previously impossible to justify. 

The Army operates a small fleet of UAVs, including Hunters and Shadows. These UAVs 
are used primarily for reconnaissance, and can fly at 60 knots or loiter at 12,000 feet for 
approximately 18 hours. A four-man team operates these vehicles: 1) a mission commander, who 
may be at a remote location from the operators at a forward base; 2) an external pilot responsible 
for taking off and landing the aircraft; 3) the AVO (aviator operator); and 4) the MPO (mission 
payload operator). 

The AVO is primarily responsible for flying the UAV, monitoring the craft parameters 
for abnormalities, attempting to correct any vehicle problems that occur during the mission, and 
following flight-path guidance from the MPO who is processing real time images. The MPO is 
responsible for finding targets, manipulating the camera to pan and zoom on those targets, 
directing the AVO to adjust flight paths to better view those targets, and detailing those targets to 
mission command. These responsibilities are extremely demanding both cognitively and 
physically. 

In the future, the Army intends to merge the responsibilities of the AVO and the MPO 
under a single operator. This single operator will also be expected to control multiple UAVs or 
unmanned ground vehicles from the same workstation—we will use the term Remotely Piloted 
Vehicle (RPV) to denote the control of any generic vehicle, whether it be air or ground. The 
challenge of controlling multiple RPVs dramatically increases the mental workload for pilots. 

The purpose of this experiment is to evaluate the capabilities of a single pilot to meet the 
multiple task demands of flying single and multiple RPVs, and to evaluate the adequacy in which 
different models of multiple task performance account for the data. In this review of the 
literature, we will analyze the mental workload demands that these responsibilities put on the 
RPV pilot. We will first discuss the general multiple RPV challenge to human cognition and 
workload modeling. Then we will analyze the interference between visually presented tasks, 
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visual monitoring, and cognitive processing. Next we will study efforts to model task 
interference via multiple resources and single channel queuing, followed by a detailed discussion 
of previous research done by others in this area. Lastly, we will focus on the current study and 
present the proposed experimental simulation. 

2.0 The General Multiple RPV Challenge to Human Cognition and Workload Modeling 

Once the responsibilities of the A VO and the MPO have been merged, a single operator 
will be required to fly the UAV, monitor the craft parameters for abnormalities, correct any 
vehicle problems that occur, manipulate the camera, find targets in the camera monitor, adjust 
flight paths to review a target area, and report back details to mission command. These 
responsibilities impose a vast amount of mental workload upon the pilot. 

Mental workload can be described as the relationship between resource supply and task 
demand. If supply exceeds demand, then perfonnance is constant. But if demand exceeds supply, 
then performance will decrease as the resource demand (workload) further increases. Each of the 
pilot’s responsibilities impose a certain amount of demand. The question is how much supply the 
pilot has available to cope with that demand, and when the demand reaches a point where 
performance drops due to a lack of resources. 

2.1 Flying the UAV 

Flying a UAV is more than just a simple tracking task. While automation does provide a 
good deal of support for flight control and stabilization, it does not always work properly 
(Parasuraman et ah, 2000) and it does not handle all flying requirements. In the following, we 
discuss generic mission requirements as they might be imposed on the single pilot of a generic 
UAV, flying without high levels of automation support. The pilot is responsible for keeping the 
plane on course and choosing new flight paths when necessary. If changes are imposed on the 
flight plan, then the pilot must manually input the new coordinates and make sure the aircraft 
proceeds as directed to the new location. If a target is selected for review, the pilot must put the 
aircraft into a loiter pattern around the target. 

While flying the UAV, the pilot is using all four stages of the human information 
processing system (sensory input, perception/cognition, selection of action, execution of action), 
involving both cognitive and physical requirements. Pilots must understand the data they are 
receiving, memorize and be able to recall those data, make decisions based on those data, and 
when course changes are required, respond by physically using the hands to manipulate the 
aircraft. 

2.2 Monitoring and correcting the craft parameters 

While flying the aircraft, the pilot must also keep track of the craft system parameters 
which are portrayed on screen. If any of these parameters becomes abnormal, the pilot must 
decide if corrections need to be made, and then make those corrections when necessary. 

Sometimes the pilot will be able to ignore the flying task while concentrating on the 
system monitoring task, but occasionally the situation may force the pilot to attempt to perfonn 
both tasks at once. This obviously creates some visual conflict, given that the sources of 


3 



information are separated, as well as cognitive conflict, since both tasks can simultaneously 
require all four processing stages, as when flying requires adjustment of a course trajectory and 
system monitoring triggers a corrective intervention. 

2.3 Target search, manipulating the camera, and reporting to mission command 

The pilot is also responsible for manipulating the camera and looking for targets. In the 
simulation examined in the present experiment, the camera pans down over the earth at a 90-150 
degree angle in a continuous live feed. If something of interest is seen, the pilot might need to 
swivel or zoom the camera for better inspection. This task can again require all four stages of the 
human infonnation processing systems, along with potentially high demands on spatial working 
memory, especially when a potential target has been found. 

Detecting a target requires substantially different processes from manipulating the 
camera. With the aircraft flying at 60 knots over a variety of terrain, it is often difficult to pick 
out exactly what is on the ground below, particularly if the pilot is also trying to simultaneously 
adjust the flight pattern or correct abnormal craft parameters on other displays. Visual scanning 
difficulties can be compounded by wide visual angles of the areas to be scanned, clutter, and 
nonsalient targets, all features common to the UAV mission. During vigilance tasks such as the 
one described in this experiment, pilots must detect intermittent, unpredictable, and infrequent 
targets, while guarding against vigilance decrements, which commonly occur even within the 
first half hour of the watch (Funk, 1991; Wickens & Hollands, 2000). In addition to detecting the 
target, the pilot must also determine what the target is, if it is of any importance, and whether or 
not to report it to mission command. 

We can easily imagine situations that involve two or more competing task conditions 
depending on what else is occurring. For example, upon detecting a target of interest, the pilot 
might have to readjust the flight plan to loiter around that target. At the same time, the pilot 
might notice an abnonnality in the craft parameters, forcing a correction, and in that scenario, 
might need to handle all three of these tasks simultaneously. By adding a second UAV to the 
scenario, these task conflicts can grow exponentially. 

Finally, the pilot must make reports (verbal responses) to mission command based on 
what is seen. If all of these task requirements are imposed at once, the mental workload on the 
pilot might be too much to allow parallel processing, forcing the pilot to revert to some degree of 
serial processing. In this case, the pilot must decide which task to do first, which next, and so 
forth; that is, to implement a task management strategy (Wickens & Hollands, 2000). One 
objective of the research is to provide data which will support a model of how the pilot addresses 
these workload overload situations. 

Clearly, one of the most important aspects of flying involves the interference between 
visually presented tasks, visual monitoring, and cognitive processing. This is a crucial issue 
when determining how much mental workload the pilot is experiencing and how well she or he 
can be expected to perfonn. The next section will focus on these issues of interference and 
highlight some of the important aspects that must be included in any workload model. 
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3.0 Interference Between Visually Presented Tasks, Visual Monitoring, and Cognitive 
Processing 

Flying a UAV can be described as a supervisory control task that frequently involves 
multiple-display monitoring. This can be further complicated by secondary tasks with higher 
level cognitive and motor components. Interference between visual monitoring and other visual 
tasks—whether these involve pure monitoring, or visual input plus higher level cognitive/motor 
tasks—is a crucial aspect of workload modeling and must be incorporated into any successful 
model. Furthermore, it is often more effective to offload some of the visual tasks to the auditory 
channel or to incorporate automation into one or more of the tasks in order to reduce visual 
workload, and total workload, respectively. This section will focus on visual-visual, visual- 
cognitive/motor, visual-auditory, and automation research in order to provide data that will 
predict multi-task efficiency under different circumstances. 

3.1 Supervisory monitoring 

Visual infonnation acquisition generally falls into one of two categories: visual search, as 
in target search described within the image display in section 2.3, and supervisory 
control/sampling, as in system monitoring described in section 2.2 (Wickens et ah, 2003) and in 
the UAV operator’s decision to deploy attention across the different areas of interest. Extensive 
research has been done on visual search (e.g., Brogan et ah, 1993; Wolfe, 1994), but we are 
equally interested in supervisory monitoring, which can be distinguished from visual search by 
four key features (Wickens et al., 2003): 1) The operator is supervising a continuously evolving 
series of dynamic processes instead of looking for a static target, 2) The operator is more 
concerned with noticing events rather than finding particular targets, 3) Measuring the proportion 
of visual attention allocated to different regions of the visual field, and the tasks supported by 
that attention, is more important than target detection RT, and 4) The operator is as much 
concerned with when to look as with where to look. 

In a model of supervisory sampling, which integrates concepts from earlier models by 
Senders (1964), Carbonnell et al. (1968), and Moray (1986), Wickens et al. (2003) proposed four 
mediating factors that drive the allocation of visual attention: 1) Salience, 2) Effort, 3) 
Expectancy, and 4) Value. This SEEV model encapsulates findings from previous research and 
models described below. 

Salience is based on the physical features of an object that make it noticeable. Examples 
of salient events might include: bright lights, loud noises, highlighted information, or auditory 
alerting (Wickens & Hollands, 2000). Making potential events within the environment salient so 
that the operator can easily find pertinent information is an example of bottom-up (stimulus- 
driven) attentional control, as opposed to top-down (goal-directed) control, which relies on the 
operator’s deliberate state of attentional readiness (Egeth & Yantis, 1997). Onsets are one 
example of stimulus-driven attentional controls that make objects more salient, as they instantly 
attract attention, particularly in the periphery of the visual system (Jonides, 1981; Remington, 
Johnston, & Yantis, 1992; Yantis & Hillstrom, 1994). 

Effort is required to move attention from one area to another. Experiments have shown 
that greater distances between visual sources produce greater deficits in performance (Martin- 
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Emerson & Wickens, 1992). Sanders (1970) described three general regions into which visual 
stimuli can fall during a monitoring task: 1) within foveal vision (0-2 degrees); 2) requiring eye 
movement (2 - 30 degrees); and 3) requiring head movement (30+ degrees). Two tasks whose 
information sources both lie within the foveal region might produce very little switching cost, 
while two tasks that appear further apart may produce more switching (Wickens, 1992; Wickens 
et ah, 2002). Martin-Emerson and Wickens (1992) analyzed visual angle separations and found a 
significant increase in event response time and tracking error for separations between a tracking 
display and a visual RT display greater than 6.4 degrees, with no performance decrements in 
either response time or tracking for visual angles below 6.4 degrees. Schons and Wickens (1993) 
also found perfonnance decrements with spatial separations between tracking and event 
monitoring displays beyond 7.5 degrees, with larger decrements in cluttered displays. Andre and 
Cashion (1993) found a linear increase in tracking error for increasing visual angle, with a large 
drop-off in performance in the head field (>56 degrees). 

Expectancy (bandwidth) was employed as a tool for modeling monitoring behavior by 
Senders (1964), and describes one part of the top-down, goal-directed movement of attention. 
Proponents of Senders’ model asserted that bandwidth was the main detenninant of monitoring 
behavior; that is, an operator would visually scan displays with high bandwidth more often than 
those with low bandwidths. Senders’ model, using eye movements in a supervisory task, 
indicates that the allocation of attention is proportional to bandwidth or event expectancy. 

The impact of expected value, or the importance of an information channel multiplied by 
the operator’s expectancy that something will occur in that channel, was first added to Senders 
model by Carbonell (1966), who turned to more realistic tasks in order to emphasize the 
importance and impact of actions taken by the operator. Carbonnell, Ward, and Senders (1968) 
found that this model predicted pilots’ scanning behavior well in a flight simulator with 
experience pilots. A subsequent model of expected value, developed by Moray, Neil, and Brophy 
(1983) and Moray, Richards, and Low (1980), which used realistic experiments involving fighter 
aircraft, predicted with reasonable accuracy what proportion of time was spent fixating on the 
aircraft and other display features. Sheridan and Rouse (1971) also followed up Carbonnell’s 
work by incorporating expected value into their model of supervisory sampling. They found that 
humans underperformed when compared to the model predictions of optimal information 
sampling. 

As mentioned, the SEEV model incorporates all of these influences into a descriptive 
model, and then uses a prescriptive model of expectancy and value to successfully predict human 
attention allocation. As with the Carbonnell and Sheridan models, expected value is considered 
the primary predictor of visual sampling, with the value of the task replacing the value of the 
visual event. Wickens et al. (2003) had experienced pilots fly in a high fidelity flight simulator, 
while engaging in traffic detection. In the first two experiments, the pilots used a cockpit traffic 
display, and in the fourth experiment they used different forms of data link displays (e.g., textual 
instructions). All three experiments showed a strong fit between model predictions and actual 
percentage of viewing time. 

Besides the four elements described in the SEEV model, there are other important factors 
to take into consideration when predicting performance in a supervisory monitoring task. For 
example, expertise will produce a more calibrated agreement between the operator’s expectancy 
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and importance (the mental model) and the true bandwidth and value of events. Moray (1986) 
notes that data from studies involving skilled pilots (e.g., Carbonnell, Ward, & Senders, 1968) 
show more optimal monitoring prescriptions than less skilled participants (e.g., Sheridan & 
Rouse, 1971), suggesting that training benefits performance in supervisory monitoring. Wickens 
et al. (2003) also suggested that well-trained pilots allocate attention more optimally than their 
novice counterparts. 

An operator’s useful field of view (UFOV) can also have effects on monitoring 
performance. UFOV is defined as “an index of the total visual field area from which target 
characteristics can be acquired when eye and head movements are precluded”. In other words, 
UFOV is the visual area that a pilot can examine without moving his/her eyes or head, while still 
procuring useful information from that visual stimulus. A pilot’s UFOV may vary depending on 
the density of visual stimuli present in the visual field (Wickens, 1990), and varies from 1-4 
degrees of visual angle (Mackworth, 1976). 

3.2 Visual + Cognitive 

UAV pilots often face a unique challenge when visual tasks do not simply involve 
monitoring. They sometimes must fly (tracking task) the aircraft while simultaneously analyzing 
possible targets, and/or diagnosing and responding to system failures (cognitive task). UAV 
pilots, in particular, may be called upon to analyze a potential target or system failure on one 
monitor, or in one channel, while not losing sight of their aviating requirements on the other. 
Adding a cognitive secondary task can lead to impairment of attention-switching abilities, 
otherwise known as “cognitive capture” (Gish & Staplin, 1995; Tufano, 1997), or even to total 
“cognitive lockup” (Moray & Rotenberg, 1986), which impedes the operator from returning 
attention to the initial primary task. Primary task performance can suffer immensely while a pilot 
focuses most, or all, of her attention on dealing with the secondary task. When designing a 
system that requires a cognitively challenging secondary task, it is important to determine 
exactly how that secondary task will affect performance in other concurrent tasks. 

Although there are a wealth of studies which show interference between cognitively 
challenging primary and secondary tasks (e.g., Lee, 1997; Moray, Richards, & Low, 1980; 
Moray, Neil, & Brophy, 1983; Moray & Rotenberg, 1986, 1989; Strayer & Johnston, 2001; 

Tsang & Rothschild, 1985), we only discuss four specific experiments because they model 
workload scenarios that are common to the UAV pilot. We believe the results from these 
experiments can be generalized to multiple-task scenarios that UAV pilots face. 

Samo and Wickens (1995) had pilots concurrently perform a tracking task, a monitoring 
task, and a decision task. The first-order pursuit tracking task resembled flying an airplane, while 
the monitoring task required the pilots to detennine when two instrument pointers went into a red 
danger zone—subjects pushed a directional button to the left or right, depending on which way 
the pointer went. The results of this experiment showed that more difficult decision tasks caused 
more interference with the primary tracking task than simple decision tasks, with greater 
disruption due to increased verbal task difficulty than with increased spatial task difficulty. 

Liu and Wickens (1992) required pilots to perform a tracking task while simultaneously 
executing either a spatial decision task (i.e., predict the future position of a vector) or a verbal 
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decision task (i.e., mental arithmetic). By manipulating the scanning distance (long / short), type 
of scanning (certain versus uncertain target locations), and decision codes (spatial / verbal), they 
were able to show that the inherently spatial visual scanning task produced more interference 
with a concurrent spatial task than with a concurrent verbal task; that is, tracking error, decision 
accuracy, and workload all suffered more when both tasks involved spatial activities. 

Wickens, Sandry, and Vidulich (1983), in Experiment 1, had pilots time share a primary 
tracking task (either first-order velocity dynamics or second-order acceleration dynamics) with a 
secondary memory search RT task responded to manually or verbally. The memory task required 
participants to respond as quickly as possible to a set of displayed letters that may or may not 
have been part of a designated memory set of characters. The results of reaction time and error 
rates suggested that manual responses disrupted tracking more than verbal responses, while 
visual inputs disrupted the memory search task more than auditory inputs; that is, increased 
perceptual competition disrupts a cognitive task more than a motor task. Furthermore, when the 
workload of the tracking task was increased by changing the dynamics from first-order to 
second-order, the advantages of the separate modalities were augmented; that is, higher tracking 
orders (i.e., acceleration dynamics) cause more interference with cognitively challenging tasks 
than with simple perceptual tasks. 

None of these three studies explicitly used an RPV-type cognitive task, yet the collective 
message derived from their results was that difficult and spatial tasks which compete for the 
same cognitive resources show greater disruption with concurrent tasks than easy or verbal tasks 
that use different resource modalities. One study which did examine an RPV-type task that was 
both spatial and difficult (but not in a dual-task context), was conducted by Gugerty and Brooks 
(2001). One of the more difficult cognitive tasks that UAV pilots must perfonn is assessing 
directional axes of buildings, or other structures, while maintaining their concurrent tracking 
duties. Gugerty and Brooks asked UAV pilots to determine which directional side of a particular 
building the parking lot with vehicles was located. They were presented with a top-down map 
which showed their current tracking direction, and a 3D egocentric map which showed the 
building and parking lots. Using the top-down map, which was always north-up, the pilots had to 
diagnose which way they were flying and then transfer this information to the 3D map in order to 
determine camera heading. A first experiment presented static maps, while a second experiment 
required the pilots to fly from one point to other in a dynamically changing environment, and 
then complete the direction task. Although Gugerty and Brooks did not examine visual-cognitive 
interference, they did note the extra difficulty of the cognitive task by pointing out that UAV 
pilots produced accuracy of less than 50% in certain camera headings (e.g., 120 or 240 degrees). 
Since we used this task in our own simulation, it is important to see how challenging this 
secondary task can be, even when performed in isolation. 

The general message offered in these studies is that concurrent cognitive side tasks cause 
more disruption of the primary tracking task than simple monitoring tasks do, particularly when 
the cognitive task is difficult, is visually displayed, involves spatial judgments, or is more 
demanding. The UAV environment described at the outset forces pilots to undergo all of these 
conditions at certain points during their missions, so it is critical to find ways to predict 
performance in situations beyond a simple visual monitoring task. 
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Fortunately, there are ways to relieve the added burden of a cognitive side task. One way 
is to introduce automation, which can be used to replace human cognition by that of the 
automation and thereby relieve task interference, particularly in visual-cognitive tasks. In the 
UAV simulation, pilots may be responsible for manually aviating and navigating their aircraft. 
Finding coordinates and making spatial judgments on flight paths consume valuable cognitive 
resources. By automating components of the tracking task, the cognitive aspects of flying are 
eliminated, reducing a visual-cognitive task to visual-visual monitoring. By removing the 
cognitive burden of the tracking task, performance in the secondary task should improve since 
visual monitoring takes up fewer overall resources than visual-cognitive tasks. The current study 
will examine the benefits of this automation by automating the tracking task in one of the 
conditions in order to analyze the effect of reducing a monitoring + cognitive task to a 
monitoring + monitoring task. 

3.3 Visual + Auditory 

Many studies have shown greater interference between two tasks that are done 
simultaneously and require processing by the same, rather than different, modality (Wickens & 
Liu, 1988). Because of this, there is interest by designers of complex systems such as UAVs in 
finding ways to offload some of the visual tasks to the auditory channel in order to reduce task 
interference. Wickens (1980) asserts, “...in a heavy visual environment, auditory displays will 
improve time-sharing performance.” In his review, Wickens cited 18 studies on dual-task 
performance and concluded that 13 of those studies showed performance in cross-modal 
configurations to be superior to intramodel configurations, particularly when the visual fields 
were widely separated in the latter. Wickens’ findings imply that the beneficial source of the 
Visual-Visual (VV) -> Auditory-Visual (AV) shift is at least peripheral in nature. 

Many driving studies take these conclusions a step further by suggesting that the benefit 
is not only peripheral, but also that different processing modalities use different cognitive 
resources, and that offloading some of the visual tasks to the auditory channel would reduce task 
interference even in non-peripheral tasks (e.g., Labiale, 1990; Liu, 2001; Parkes & Coleman, 
1990; Streeter, Vitello, & Wonsiexicz, 1985). For example, Gish, Staplin, Stewart, and Perel 
(1999) had participants perform a primary driving task while dealing with a secondary cognitive 
task of reading or listening to a set of instructions, and found that driving performance, (i.e., the 
percentage of correct braking responses to traffic events), and the secondary task performance 
(i.e., the percentage of correct responses to the navigational task) were both better under the 
auditory condition than under the visual condition. 

It is important to note that offloading visual tasks to the auditory channel does not always 
produce better perfonnance on both tasks. A number of investigations involving a continuous 
tracking task combined with a discrete reaction time task have shown that, on average, 
offloading to the auditory channel improves perfonnance on the discrete task whose modality is 
changed, while either showing no benefit to, or even reducing, perfonnance in the visual 
tracking task (Israel, 1980; Lee, 1997; Pamperin & Wickens, 1987; Tsang & Wickens, 1988; 
Wickens, Braune, & Stokes, 1987; Wickens, Dixon, & Seppelt, 2002; Wickens & Goettl, 1984; 
Wickens, Vidulich, & Sandry-Garza, 1984). This phenomenon may be attributed to a kind of 
auditory preemption, wherein the auditory channel “grabs” the operator’s attention away from 
the primary task to focus on the secondary auditory task. 
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Furthermore, offloading to the auditory channel may sometimes decrease performance in 
both the tracking task and the secondary task (e.g., Matthews, Sparkes, & Bygrave, 1996; 

Stanton & Baber, 1997; Tsang & Rothschild, 1985). Helleberg and Wickens (2001) presented air 
traffic control (ATC) instructions to participants either visually or aurally and found that the 
primary task (traffic monitoring and flight path tracking) suffered as well as the secondary task 
(accuracy of communications task) from auditory delivery of ATC information. 

As Wickens and Liu (1988) pointed out, it is sometimes difficult to predict exactly when 
offloading to the auditory channel will improve perfonnance. Wickens, Dixon, and Seppelt 
(2002) point out that visual angles and preemption can moderate the auditory benefit during a 
VV to AV shift. The greatest benefit to auditory offloading comes when the visual-visual 
conditions require widely separated displays, so that auditory offloading eliminates visual 
scanning. On the other hand, auditory delivery may help the secondary task, while disrupting the 
primary task, because of an auditory preemption effect of the secondary task. 

fn summary, UAV pilots are often faced with supervisory control situations that involve 
monitoring multiple workstations or displays, resulting in problems of dual-tasking. These 
problems can be further exacerbated by cognitively challenging secondary tasks which may 
result in cognitive capture or lockup. Automation may eliminate some of the cognitive demands, 
thereby reducing overall task load, and restore higher levels of dual-task performance. Changing 
information delivery to an auditory channel may accomplish the same goal by redistribution of 
task load across separate resources, particularly in the UAV environment where visual channels 
are widely separated. The findings above have generated different theories regarding why this 
dual-task interference happens (e.g., Craik, 1948; Kahneman, 1973; Moray, 1986; Navon & 
Miller, 1987; Pashler, 1994; Temprado et al., 2001; Wickens, 1980). fn the next section, we will 
discuss different workload models, postulating different psychological mechanisms, that have 
been generated from three of these theories, if validated by the empirical data reviewed above, 
and to be reported in the experiment described below, these workload models can be used to 
predict performance in future dual-task situations. 

4.0 Three Workload Models 

Having a quantitative workload model is extremely useful to system design, as it allows 
system engineers to compute the capability of humans to operate machines before any time or 
money is spent on developing those machines (Laughery & Corker, 1997; Pew & Mavor, 1998; 
Wickens, Vincow, Schopper, & Lincoln, 1997). Workload models can usually be categorized in 
one of the following two ways: models derived from the Single Channel Theory which consider 
tasks too difficult to allow time-sharing, and models derived from Single or Multiple Resource 
Theory which assume that some conditions will allow parallel processing or concurrent 
processing (Sarno & Wickens, 1991). 

The resource models are based on the concept that an operator has a limited capacity of 
resources and that different tasks require varying amount of resources (Kahneman, 1973; Moray 
1969). When the task demand exceeds this finite capacity, it results in some task interference, to 
the degree that is related to the difficulty (resource demand) of the task (single resource theory) 
and/or to the competition for overlapping or shared resources (multiple resource theory). For a 
workload model to be successful, it must first be a good task interference model; that is, predict 
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when the perfonnance in one task will suffer as more resources are demanded for another task, 
or as structural changes are made to one of both tasks. 

4.1 Single Channel Theory (SCT) 

The origins of single channel theory (SCT) can be traced back to the early postulations of 
the “single channel bottleneck” in the human infonnation processing system. Proponents of this 
theory claim that two high speed tasks can not be performed concurrently, but will result in total 
abandonment of all (or part) of one task until the other task is completed (Broadbent, 1958; 

Craik, 1948; Welford, 1967). SCT assumes that no parallel processing or timesharing can take 
place and that all tasks are performed in a series. Time is a limited resource (Hendy, Liao, & 
Milgram, 1997) and can not be shared across tasks. Some experiments have presented two visual 
tasks (Neisser, 1969) as well as two auditory tasks (Cherry, 1953; Glucksberg & Cowen, 1970; 
Moray, 1959; Nonnan, 1969; Mowbray, 1964) and appear to provide results that agree with the 
SCT model, in that successful perfonnance in one task entirely precluded success in a concurrent 
task. 


The existence of the psychological refractory period, or PRP (Telford, 1931; Kantowitz, 
1974, Meyer & Kieras, 1997; Pashler, 1994), also appears to support SCT assumptions. The PRP 
describes a situation in which two RT tasks are presented close together in time (Wickens & 
Hollands, 2000, p. 367). PRP experiments (see Pashler, 1994, for a review of the literature) 
indicate that the processing of infonnation for one task mandates a delay in processing the 
stimulus for a second task that arrives just after the first task, with that delay increasing as the 
two tasks are presented closer in time. Pashler (1994) points out, however, that while the original 
bottleneck theories assumed a perceptual bottleneck, his later studies show this bottleneck to 
actually be in the response selection stage. 

Much of the evidence for single channel theory comes from the basic laboratory research 
of the PRP paradigm. However, other work has provided evidence for single channel behavior in 
more complex task environments. As an example, Hendy et al. (1997) analyzed task interference 
in an air traffic control environment. Using a low-fidelity simulated radar screen, participants 
controlled the altitude and direction of aircraft moving across their field of vision. The goal was 
to route the “aircraft” successfully to their destinations in the least amount of time. By 
manipulating the number of aircraft and the length of the update interval (how often new 
information appeared on the screen), the researchers attempted to show that humans can only 
perform one task at a time. They found that for the dependent variables error, success, and 
correct, a time-based model more consistently predicted the results than did an intensity-based 
(resource) model. However, the authors clarify their conclusions by stating that they are not 
returning to a strictly single-channel model of traditional timeline analysis, but rather that 
concurrent processing is assumed when different processing structures come into play. 

Moray and Rotenberg (1986) cited “cognitive lockup” as one reason for single channel 
processing. In their study, participants were responsible for monitoring four simulated thennal 
hydraulic systems, similar to Crossman and Cooke’s (1974) bathwater task. After a period of 
time, one of the systems would fail (i.e., a blockage in an outflow pipe), resulting in a decrease 
of flow through the valve. In the following trial, a second system would fail 75 seconds after the 
first system failed. The data showed that the time participants took to detect the second failure 
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was much longer than the time to detect the first failure, suggesting that cognitive lockup while 
dealing with the first failure had prevented the operator from effectively monitoring the rest of 
the systems and noticing the second failure. Moray and Rotenberg reasoned that people tend to 
deal with problems serially, rather than switching between tasks. 

Kerstholt et al. (1996) extended this paradigm by introducing an additional behavioral 
response that would allow participants to stabilize a faulty subsystem temporarily while they 
dealt with another fault. If the operator ignored this optimal strategy, in favor of resolving the 
first problem completely before moving onto the second problem, it would stand to reason that 
humans prefer strict serial processing over more rapid switching even when it leads to 
suboptimal behavior. Their supervisory control task involved visually tracking four subsystems, 
diagnosing problems when they occurred, and correcting those failures before the system went 
into automatic shutdown. Their general results agreed with their “cognitive lockup” hypothesis; 
that is, most participants failed to use the temporary stabilization option. However, individual 
subject data varied greatly, and some participants showed optimal behavior not consistent with 
cognitive lockup. 

Since SCT offers a baseline performance measure, it is an excellent theory with which to 
compare to other theories. However, while the studies highlighted above appear to validate SCT, 
weaknesses in the theory show up in experiments not specifically designed to evoke single 
channel behaviors (e.g., Meyer & Kieras, 1997), as well as basic laboratory research explicitly 
designed to disprove SCT (e.g., Schumacher et al. 2001). Samo and Wickens (1995), mentioned 
previously, compared their data to models predicted by SCT, and found no correlation between 
predicted and obtained decrements. They rejected the time-based model in favor of resource 
models which were more sensitive to the resource demands of the tasks involved and better 
predicted differences in the obtained dual-task performance data. 

In summary, SCT has different manifestations. All versions of strict SCT predict that 
progress on information processing can only take place on one task at a time, and therefore the 
completion time for two tasks imposed concurrently will equal the sum of the completion times 
for each done alone. This concurrent completion time will increase to the extent that information 
for a second arriving task is closer in time to the initiation of the first arriving task. These two 
predictions will be examined in the current research. 

Within this class of strict SCT versions, predictions may vary as a function of how 
frequently attention (and therefore processing) is switched between the two tasks. If the switch is 
only perfonned once, such that the first arriving task is fully completed prior to beginning 
processing of the second task, then the form of “cognitive tunneling” or “lockup” described 
above, is observed. However, more rapid switching will allow both tasks to absorb the single 
channel delay, even as the sum of their completion times adheres to the strict SCT predictions. 

Other less strict versions of SCT assume that there is parallel processing at earlier stages 
(Pashler, 1994). However, a major focus here will be on situations in which early processing will 
be likely to be single channel because it is based on widely spaced visual inputs which cannot 
support concurrent processing (Moray, 1986). 
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4.2 Single Resource Theory (SRT) 


Single resource theory (SRT) differs from SCT in that task interference prediction is a 
function of cognitive resources rather than the amount of time available. Resource theory allows 
for concurrent tasks, or parallel processing, while strict SCT does not. Moray (1967) developed 
the concept of a “limited capacity central processor” which gives humans the ability to share 
resources between tasks in certain situations. This concept evolved into a model of attention by 
Kahneman (1973) and applied work by Rolfe (1973). Their models equated the attentional 
resources with “mental effort” and enabled concurrent processing of the tasks. While generally 
predicting less effective concurrent task performance with greater task difficulty, the models 
allowed motivation, and a subsequent mobilization of more effort, to partially overcome the 
penalties of increased task difficulty (Vidulich & Wickens, 1985). 

Resources, whether physical or mental, can be strategically allocated to different tasks 
when needed (Wickens, 2002). If one task is very simple and requires almost no resources (e.g., 
tapping ones lingers on the table), then it is “data limited” (Norman & Bobrow, 1975), and 
ample resources can be allocated to another concurrent task. For example, a pilot might take 20 
seconds to adjust the throttle. This same pilot might also take 10 seconds to bank an airplane. 
Under the assumptions of strict single channel theory, these two tasks cannot be done in less time 
than the sum of the two tasks; that is, it will take 30 seconds to perform both tasks. But under the 
assumptions of resource theory, the easier task (adjusting the throttle) should consume fewer 
resources than the more difficult task (banking the airplane). Adjusting the throttle is an almost 
automated task, and these types of tasks may demand very few resources (Fitts & Posner, 1967; 
Schneider, 1985). In this scenario, while simultaneous execution may produce some degraded 
performance of one or both tasks, the operator should be able to do both tasks in less than 30 
seconds—the amount of time required as predicted by single channel theory—and will probably 
complete them both in 20 seconds, the amount of time of the longest task. That is, the time 
required to do two tasks concurrently is less than the sum of the two single task times, and 
interference will result from their concurrence, not the postponement of one task or the other. 

4.3 Multiple Resource Theory (MRT) 

The multiple resource theory (MRT) proposed by Kantowitz and Knight (1976), Navon 
and Gopher (1979), and Wickens (1980) expands upon single resource theory by making the 
assumption that variance in dual task performance is not simply attributed to the difficulty 
(quantitative resource demand) of the components, to the levels of automaticity, or to the 
resource allocation strategy between them (i.e., choosing to emphasize one task more than 
another) (Wickens, 2002), but also to the differences in time sharing efficiency that support the 
concept of separate resource structures (Kantowitz & Knight, 1976; Wickens, 1980). As 
mentioned previously regarding visual-auditory presentations, two tasks which utilize different 
resource structures are performed more efficiently than two tasks which use the same resource 
structures (Kantowitz & Knight, 1976; North, 1977; Wickens, 1980), and can sometimes result 
in almost perfect time-sharing (Allport et ah, 1972; Schumacher et ah, 2001). For example, 
driving while listening to the news on the radio is much easier than driving while reading the 
same information in print. Driving and reading both use visual resources, while driving and 
listening to the radio use separate visual and auditory resources. Performing two visual tasks 
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(VV) simultaneously often results in poorer performance than performing an auditory and visual 
task (AV) (see section 3.3 for studies which support this view). 

The 4-dimensional multiple resource model proposed by Wickens has four categorical 
and dichotomous dimensions (Wickens & Hollands, 2000): 

1. Processing stages - resources used for perceptual and cognitive activities differ from 

those used for selection and execution of responses (e.g., Israel, Chesney, Wickens, & 
Donchin, 1980; Israel, Wickens, Chesney, & Donchin, 1980; Shallice, McLeod, & 
Lewis, 1985). For example, when driving a vehicle, the operator can execute 
maneuvers while simultaneously perceiving other vehicles around her. 

2. Perceptual modalities - as we have noted above, it is sometimes easier to divide 

attention between the eye and ear than between two visual or two auditory channels 
(e.g., Parkes & Coleman, 1990; Rollins & Hendricks, 1980; Seagull, Wickens, & 
Loeb, 2001; Wickens, Sandry, & Vidulich, 1983). For example, is easier for pilots to 
listen to instructions while scanning the outside world than to read information off a 
display while performing the same scanning task. 

3. Visual channels - focal and ambient vision appear to be associated with different 

resource structures (Leibowitz, Post, Brandt, & Dichgans, 1982; Previc, 1998; 
Weinstein & Wickens, 1992). For example, it is quite common for people to foveate 
on a target while processing other peripheral visual stimuli when walking, driving, 
flying, etc. 

4. Processing codes - there appears to be a distinction between analog/spatial processes 

and categorical/symbolic processes (Poison & Friedman, 1988) particularly between 
manual and vocal responses (Martin, 1989; McLeod, 1977; Tsang & Wickens, 1988; 
Vidulich, 1988; Wickens, 1980; Wickens & Liu, 1988; Wickens, Sandry, & Vidulich, 
1983; Sarno & Wickens, 1995). For example, one can draw a map while 
simultaneously explaining directions. 

Both SRT and MRT are extremely useful when predicting perfonnance in different dual-task 
conditions. Their particular strengths lie in predicting the changes in dual-task interference that 
might be brought about by making changes in task difficulty (SRT) and in the interface (MRT), 
while SCT is limited to task interference predictions based only on the amount of time the 
different tasks consume. 

There are clearly some circumstances in which SCT adequately accounts for data, 
particularly in some RT tasks (see Schumacher et ah, 2001, for an exception), conditions with a 
dedicated need for foveal vision and widely separated visual displays, possible overload of 3- or 
4-task combinations (Liao & Moray, 1993), and possibly the circumstances that trigger 
“cognitive tunneling” as discussed in Section 3.2 above. We hypothesize that one of the 
circumstances that may be more likely to be modeled by SCT is the multiple workstation 
environment typical of the UAV scenario discussed at the outset. In order to examine this 
hypothesis, we will now review the small set of studies that have examined time-sharing across 
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multiple workstations, with particular emphasis on those which might have tried to model the 
dual (or multiple) task performance breakdowns. 

5.0 Multiple Workstations 

Although there is a wealth of data on multi-channel monitoring, some of which has been 
described above, there appear to be very few studies in the literature that have addressed the 
monitoring of multiple systems, and in particular, multiple replicas of identical systems (such as 
the 2 UAV problem described at the outset). Furthennore, fewer still have examined the 
workload implications of increasing the number of such systems. 

As one example, Murray and Caldwell (1996) manipulated the number of displays to be 
monitored and found significant perfonnance penalties in multiple-platform controls; that is, 
even a moderate number of displays produced response time penalties, evident at relatively low 
levels of complexity. However, Murray & Caldwell did not use their results to test workload 
theories (e.g., no single channel model tests), which prevents full relevance from being observed. 
Furthermore, the tasks associated with each display were relatively simple compared to those 
involved in UAV control. 

A search of the literature failed to locate any additional studies that examined operator 
supervisory control of multiple replicas of the same system. However, two studies will be 
considered in some detail that examined supervisory control of multiple complex tasks, 
displayed in visually separated locations. In one approach approximating the complexity of a 
multiple-UAV workstations, Liao and Moray (1993), created an experimental paradigm 
consisting of four tasks: 1) manual compensatory tracking task, 2) “radar-like” monitoring task, 
3) choice reaction time (CRTT) task, and 4) mental arithmetic task. They also created a 
quantitative model that they feel is partially validated and predicts performance adequately 
enough to be practical and useful in most system design scenarios. Their calculations were all 
programmed into MicroSAINT, a software modeling simulator. If SCT were correct, then 
multiple-task experimental performance measures would equal those predicted by the model 
simulation. Furthermore, a successful validation of SCT would help to constrain the domain of 
applicability of other competing theories. 

During the first experiment, which was used to create the model, these four tasks were 
performed individually (single-task condition) and then all four were carried out simultaneously 
(4-task condition). During this experiment, the model was adjusted to fit the data. During the 
second experiment (validation of the model) the tasks were done in pairwise conditions. The 
model, having been adjusted for fit during the first experiment, was measured directly against the 
data from the second experiment in order to validate the model. 

Their findings reveal that the simulation model did rather well in predicting some single 
task conditions. In the 4-task condition, the simulation model once again predicted most of the 
performance measures. Overall, this appears to be a good fit, but it is important to add that the 
subjects were forced to remove their hands from the joystick when making responses. This 
naturally forces single-channel processing, because it is impossible to continue tracking without 
holding the joystick. Furthermore, as mentioned previously, the simulation model in this first 
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experiment was adjusted to better measure actual experimental perfonnance, bringing into 
question the integrity of the “fit” between the model and the experimental results. 

To properly validate the model, the authors conducted two more experiments, whose 
results were compared directly to the finished model without further adjustment. Unlike the first 
experiment which involved a four-task scenario, these validation experiments involved dual-task 
scenarios. The authors report that the model did a fair job in predicting overall perfonnance. 
However, it overpredicts tracking error in the first validation experiment, as well as 
overpredicting CRTT in the second validation experiment. As the authors point out, actual 
performance of the tracking task and CRTT task was better than the model predictions, 
indicating that some parallel processing had occurred. 

As noted, the two validation experiments in the Liao and Moray study only involved 
dual-task conditions. As seen in the Sarno and Wickens (1995) study mentioned previously, 

MRT typically does a better job of predicting dual-task performance than SCT does, particularly 
when displays are close enough together to facilitate parallel processing (e.g., eliminate the 
single channel bottleneck created by the need to access foveal vision), or when certain tasks are 
offloaded to other non-visual channels. Even though Liao and Moray made no attempt to offload 
some of the visual tasks to the auditory channel, their data still indicate some parallel processing, 
which would be predicted under the resource models. Offloading to other channels might have 
resulted in even more parallel processing. 

Since it appears to predict performance well in some multiple-task conditions, clearly the 
single channel model is the baseline model against which we can compare more sophisticated 
resource models to understand how much added variance they can account for in dual task 
performance. 

The second study we review in depth employed the same paradigm used in the current 
research. Wickens and Dixon (2002) looked at many of these workload issues surrounding 
auditory offloading and automation in a high-fidelity UAV environment and compared their 
results to all three interference models. They had pilots fly a UAV through a series of mission 
legs designed to replicate a typical Anny reconnaissance mission. During the mission, the pilots 
were responsible for flying from one command target to the next (using a 2-D top down map), 
while simultaneously searching for, and analyzing, targets of opportunity (TOO) in a 3-D 
egocentric camera display, and monitoring system parameters for possible failures. They were 
also required to remember flight instructions presented in a message box at the beginning of each 
leg, and could refresh their memory at any time by retrieving the infonnation. 

One condition, using all manual control and visual displays, measured baseline 
performance in tracking error, system failure (SF) detections and response times, TOO detections 
and response times, and recall of the flight instructions. Another condition provided auditory 
alarms which alerted the pilots to SFs represented by slowly oscillating vertical bars that 
occasionally went “out of bounds”, as well as auditory flight instructions, offered through 
synthetic voice. A third condition offered automated flight control, which relieved the pilot of 
having to manually fly the aircraft. 
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The authors reported that auditory alarms were beneficial in improving detection rates as 
well as reducing response times to SFs that occurred concurrently with tasks which were in the 
easy to medium difficulty range (e.g., orienting UAV, nonnal flight, searching for TOOs), but 
not with those that occurred concurrently with highly challenging tasks (e.g., target inspection). 
Cognitive lockup was cited as a possible cause for this apparent lack of parallel processing; that 
is, despite the pilots ears having access to the auditory tone that alerted them to a SF, they 
appeared not to divert their attention to address the failure, while engaged in the most heavily 
demanding tasks. However, it was unclear whether the lockup was strategic (pilots’ choice) or 
structural. Analysis of the memory task revealed improved performance in the auditory condition 
due to increased parallel processing; that is, offloading one of the visual tasks to the auditory 
channel allowed pilots to perfonn both tasks concurrently rather than switch back and forth, as 
was necessary when all tasks were displayed visually. 

In contrast to the auditory offload, automated flight control not only improved tracking, 
but it also improved detection of the TOOs, as well as detection of the SFs which occurred 
during the early stages of a mission (i.e., orienting UAV, nonnal flight). An added bonus was 
reducing the number of repeats for the flight instructions, as pilots no longer had to remember 
fly-to coordinates for the next waypoint. 

Differences in perfonnance between simple dual task combinations (e.g., flight control 
and SF monitoring) and difficult dual task combinations (e.g., target inspection and SF 
monitoring) can be explained by resource models due to changes in task difficult. On the other 
hand, if the difference between easy and hard tasks is characterized by time differences, then 
these results can also be predicted by SCT. The finding of improved performance via auditory 
offloading can only be predicted by multiple resource models, as the other two models discussed 
do not make such predictions. The apparent cognitive lockup found when SFs were combined 
with target inspection appeared to be predicted by the single channel model. 

6.0 The Current Study 

The current experiment was designed to examine the UAV paradigm used by Wickens 
and Dixon (2002), in both single (replicating Wickens & Dixon) and dual UAV control 
conditions. In addition to doubling workload in the dual UAV condition, the second primary 
difference between this, and the previous study, was the addition of motivational incentives for 
good perfonnance, to detennine if such incentives, by inducing a greater mobilization of effort, 
could mitigate the negative effects of cognitive tunneling observed in the previous study. 

To review the paradigm used in the previous study, pilots performed three primary tasks: 
(1) meeting the mission goals of “fly to and report”, (2) monitoring for TOOs and reporting 
them, and (3) monitoring for system failures and conecting them. It is noted that each of these 
three tasks, in turn can be subdivided into 2 or 3 phases. (1) Mission goals: (i) read and initiate 
trajectory, (ii) monitor flight path, (iii) loiter, zoom and describe command target. (2) TOO task : 
(i) monitor 3D image display, (ii) loiter, zoom and describe TOO. (It will be noted that the loiter 
phase of tasks 1 and 2 is identical in its demands). (3) SF task: (i) monitor SF display, (ii) 
identify SF, type in identity and current coordinates. A subtask of Mission goals, involved 
exercising the opportunity to retrieve the CT instructions if necessary. Pilots also performed the 
three tasks under each of three conditions: a) baseline, b) auditory offload in which CT 
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instructions and SF’s were presented auditorally, and c) automation offload, in which flight path 
tracking was automated after the CT coordinates were entered. 

Table 1 presents the four tasks, and three conditions, along with a third dimension which 
describes the estimated task demands of each, along the three stages of infonnation processing, 
perception, cognition and response. Each stage is associated with an experimenter-assigned 
demand value, assigned on the basis of the least integer ordinal heuristic, by which ordinal 
relations between demand values are preserve, but the smallest integer (or half integer) values 
possible are chosen (Wickens, Helleberg, Horrey, Goh, & Talleur, in press). This produces a 
range in values between 0 and 2. On this basis, it is possible to identify conflict points of 
concurrent task requirements between the three major tasks and their subtasks, when the mission 
time line of possible task overlap is presented. This time line is shown in Table 2, which 
describes the sequence of possible activities within each leg. Where there is no task component 
active, the cell in the table remains blank. Where there is an active task component, the demand 
values shown are based on the average demand value, across all three stages within the baseline 
condition as shown in Table 1 (Corresponding tables can be constructed for the AUD and ATM 
offload conditions). As such, a total interference score, based upon single resource theory, can be 
computed simply by summing the demand values of all active tasks at a given moment. For 
example, during the “initiation” phase of each leg, a prediction of interference level would be 2.0 
(1.7 + 0.3), an identical value to that observed during forward flight, (1.0+0.7+0.3), which 
requires an added task (TOO monitoring) but reduced demands on the mission task because the 
flight trajectory does not need to be selected, only monitored. It will be noted that the SF task 
row actually contains two demand levels, one, for SF monitoring, which is always active, unless 
a SF occurs, and the other, for SF-reporting, that is only active on those phases when a SF occurs 
and is detected. 
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Table 1. Demand values in perceptual, cognitive, and response stages for the different tasks and 
conditions. 
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* Flying demands for ATM are reduced because all that is necessary is to monitor position along 
the flight path, not deviations from it. 

# Auditory perception of CT instructions are reduced because pilot can listen to them while 
concurrently focusing attention on the nav display to find the required coordinates. 

@ Cognitive demands of CT instruction retrieval are reduced in the automation condition, 
because pilot only needs to retain information about the report requirements, not the CT location. 

+ Perceptual demands of monitoring are reduced because of automatic auditory onset capture. 


Table 2. Active concurrent tasks. Estimated demand values are shown. The values for SF 
monitoring and SF report are offset, to indicate that these cannot occur concurrently. The “?” for 
CT infonnation retrieval reflects the uncertain pilot-dependent aspect of this task. 

PHASE OF MISSION LEG 


TASK 

Initiate 

Fly 

TOO Inspect/Renort 

Fly 

CT Inspect/ Report 

Mission 

1.7 

1.0 

— 

1.0 

2.0 

TOO 

— 

0.7 

2.0 

0.7 

— 

SF-Mon 

0.3 

0.3 

0.3 

0.3 

0.3 

SF Report 

1.3 

1.3 

1.3 

1.3 

1.3 

Retrieve CT? 

— 

0.7 

— 

0.7 

0.7 
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On the basis of the data presented in Tables 1 and 2, it is possible to make a number of 
predictions regarding the effects of the different conditions. Based upon a pure SRT model, we 
predict that the level of performance decrement will be directly related to the number of “active 
tasks” as defined by the cell entries in Table 2, and how these entries may be deleted by auditory 
and automation offloads.. However additional tests of single channel theory can be made on the 
basis of more detailed time line analysis that considers the delay in processing concurrently 
performed tasks, and how this delay is influenced by arrival time differences.. 

Based upon single resource theory, performance decrement levels can be predicted by 
summing the demand values within each column of Table 2 defining a task phase, as described 
above. 


The predictions from multiple resource theory are somewhat more complex, as these 
require integrating the active task combination matrix in Table 2, with the stage-defined resource 
demands of Table 1, and incorporating additional consideration of the differences between 
auditory and visual resources, and between verbal and spatial resources, not described in Table 1. 
These features are represented in modeling based upon the IMPRINT software, described later in 
this report. However, the most important predictions of MRT are related to the changes in task 
interference brought about by auditory offloading of SF monitoring, and CT information. Both of 
these changes are predicted by MRT to reduce the interference with all other visually displayed 
tasks during the times when the SF monitoring task is active, and during those pilot-chosen 
instances in which the CT infonnation is retrieved. 

Finally, we note that the above task representations and their predictions apply to single 
UAV flying. We predict that they should apply to dual UAV flying as well. Furthermore, two 
additional model-based predictions can be made in transitioning from the single to the dual UAV 
conditions. First, predictions from strict single channel theory would dictate that, since twice as 
much time is occupied in dual UAV flight, all of the decrements should be doubled, and 
interference should be “massive” (e.g., the decrement caused when comparing baseline to 
automation in single task conditions should be doubled in dual task conditions). Second, the 
version of single resource theory proposed by Kahneman, is one that suggests that as demand 
grows, the capacity to supply additional resources to meet that demand will diminish. Hence any 
changes causing increased interference in single UAV conditions (e.g., going from automation to 
manual flight control), should cause substantially more interference in dual UAV conditions. 
Multiple resource theory does not make either of the above predictions. 

Methods 

Participants 

Participants were 34 male and 2 female undergraduate students (ages 18-25) enrolled in 
the University of Illinois Aviation Program. All the participants had at least a private pilot’s 
license, with some instrument flight experience. All participants received $8 per hour for their 
time. Participants were motivated to perform at their best with rewards of $10 and $5 for 1 st and 
2 nd place finishes, respectively, in their group of six pilots. 
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Apparatus 


The apparatus for the individual tasks was almost identical to Wickens and Dixon (2002). 
An Evans and Sutherland SimFusion 4000q, with dual 1.0 Ghz processors and an OPENsim 
Graphics card generated the UAV simulation. Each of the two UAVs was displayed on separate 
Hitachi CM721F 19-inch monitors, using 1280x1024 resolution. Figure Ml presents a sample 
display for a single UAV and Figure M2 shows the visual angles between tasks. 



3-D Image Display 

Search for targets 

Zoom & inspect these 
when found 


Message Box 

Fly to coordinates & report question 


2-D Navigational Display 

Track UAV to reach waypoints 


System Gauges 

Monitoring task 

Identify failure when 
it occurs 


Figure Ml. A screenshot example of the experimental display with verbal explanations for each 
task window. Actual display was larger and more legible than this rendering. 



Figure M2. Range of visual angles between the four main windows / tasks. The ranges go from 
the two farthest points of interest to the two closest. The average visual angle is equal to the 
visual angle between the center points in the diagram. 
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As seen in Figure Ml, the experimental environment was subdivided into four separate 
windows for each UAV. Figure M2 contains the range of visual angles between the individual 
windows. The top left window contained a 3D egocentric image view of the terrain forward 
and/or below the RPV. The sample figure shows a command target (CT) at normal viewing 
distance (i.e., 6000 feet altitude). The ability to manipulate this view depended on whether the 
operator was tracking a straight line path or loitering around a target. During regular tracking 
periods, the operator could view straight down to the ground or pan the camera up 45 degrees 
towards the horizon by manipulating the camera’s y-axis. The momentary camera angle was 
indicated by a yellow dot along a red y-axis bar at the bottom center of the 3D window. During a 
loiter pattern, the operator was able to extend the viewing angle from 0 to 90 degrees along both 
the x- and y-axes. A zoom feature (up to lOOx) was also available only in the loiter pattern. 

The bottom left window contained a 2D top-down map of the 20x20 miles simulation 
world. Coordinates (which formed a grid) from 0-100 were placed along the x- and y-axes for 
navigation purposes. The yellow and red lines denoted minor and major roads, respectively. The 
smaller blue lines denoted rivers, and the large blue shapes denoted lakes. 

The bottom center window contained the Message Box, with “fly to” coordinates and CT 
report questions. These instructions were present for 15 seconds at a time. During the 
Automation condition, the lower half of this box also contained a place to type in the coordinates 
for the next CT. 

The bottom right window contained the four system failure (SF) gauges. Each gauge 
represented a different onboard system. The white bars oscillated up and down continuously, 
each driven by sine waves ranging in bandwidth from 0.01 Hz to 0.025 Hz. A SF occurred when 
one of the white bars moved gradually into a red zone. 

Participants used a Logitech Digital 3D joystick to manipulate the aircraft/camera and a 
X-Key 20-button keypad with which to indicate responses. Each UAV had its own joystick and 
keypad. As seen in Figure M3, the joystick had controls for turning the UAV, manipulating the 
camera on the x- and y-axes, zooming, detecting targets, loitering around targets (to the left or 
right), and detecting SFs. The keypad was used for indicating which system failure occurred, the 
ownship coordinates for that system failure, and for typing in mission coordinates during the 
Automation condition. The experimenter used a separate keypad to record correct or incorrect 
responses and to indicate when the operator detected a target of opportunity (TOO) or a 
command target (CT). 
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loom 


Detect SF 


Trigger 

(detect targets) 

Loiter 


Figure M3. Logitech joystick and X-Keys keypad. 


Conditions and Tracking Task 

There were three conditions: 1) Baseline, 2) Auditory, and 3) Automation. In each 
condition, the participants were seated approximately A meter away from the screen and 
required to fly two missions, each of which consisted of 10 legs with command targets at the end 
of each leg. One mission involved flying a single UAV and the other mission involved flying 
two UAVs simultaneously. Figure M4 shows a sample mission, highlighting the command 
targets and flight paths of the 10 legs. Each leg was approximately 10-12 kilometers long, and 
took approximately 5-8 minutes to complete. Each mission used a different set of legs and 
targets, so there were no repeating maps or targets for a participant. 
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Figure M4. A sample mission. CTs are designated with red X’s while TOOs are designated with 
red O’s. This preplanned route was not shown to participants. The wide path along each leg 
indicates the width of the image in the 3D view when at minimum zoom. 


There were two different forms of tracking control: 1) manual mode, and 2) automatic¬ 
mode. During the Baseline and Auditory conditions, the participants were required to manually 
control the UAV heading through each mission. This first-order control was accomplished by 
twisting the joystick to the left or the right. There was no disturbance in the control; that is, if left 
alone, the UAV would travel in the straight line established by the twist without deviation from 
its path. Participants were not responsible for airspeed (fixed at 70 knots), or altitude (fixed at 
6000 feet). The operator did not have the capability to pitch, bank, or roll the UAV. 

During the Automation condition, the operator was not responsible for manually tracking 
the UAV. Instead, he or she was required to type in the mission coordinates of the next command 
target at the beginning of each leg, using the keypad. The computer then automatically guided 
the UAV along a direct, straight-line path to those coordinates. 

Both the Baseline and Automation conditions entailed visually reading all instructions 
and system parameters. The Auditory condition presented auditory instructions and alarms for 
system failures (see below). 
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Target Searching and Reporting Tasks 

A command target (CT) was located at the end of each mission leg, at the coordinates 
specified at the leg beginning. As seen in Figures Ml and M5, which depict a typical CT at Ox 
zoom and again at lOOx zoom, respectively, these were very salient and easy to find. They 
consisted of a building (e.g., warehouse, factory, hanger, etc.) with 1-3 tanks and/or helicopters 
located within 10-50 feet around them. These weapons were always located on the north, south, 
east, or west sides. 



Figure M5. An example of a command target under a lOOx zoom, from an angle looking straight 
down. 


The pilots were required to loiter around all CTs, zoom in the camera for a closer view, 
and respond to questions that appeared in the message box (or were spoken in the Auditory 
condition) about what they could see. Sample questions might be: 1) How many tanks are there 
and where are they located in relation to the building?, 2) Report the number of weapons present, 
or 3) Where are the helicopters located? These questions could only be answered once the 
operator had zoomed in close to the CT. 

These questions were offered once at the beginning of each leg and stayed visible in the 
message box for 15 seconds (in the manual and automatic modes) or were presented aurally by 
digitized speech in the auditory mode, lasting a duration of approximately 7-9 seconds. If the 
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pilots forgot the question, they were allowed to hit a Repeat button on their keyboard at any time. 
The number of repeats was recorded. 

Along each leg, pilots were also instructed to search for Targets of Opportunity (TOO). 
Figures M6 and M7 depict a typical TOO at Ox zoom and again at lOOx zoom, respectively. As 
seen in Figure M6, these TOOs were camouflaged and difficult to see at 6000 feet (i.e., zero 
zoom), and could not generally be detected unless foveated. All TOOs were the same basic 
square “bunker” shape and came in three sizes, with areas of approximately 1, 1.5, and 2 degrees 
of visual angle at zero zoom. There was one TOO per mission leg, which was located randomly 
somewhere in the middle 60% of each leg (i.e., between 20% and 80% of distance traveled); 
however, participants were not told this. They were only told that the TOO was somewhere 
along the direct-line path between CTs. Around each TOO were 1-3 tanks and/or helicopters, 
located within 10-30 feet of the bunker. These weapons were always located on the north, south, 
east, or west sides. 



Figure M6. An example of a medium-sized TOO at Ox zoom. The TOO is located just below the 
arrow. On the actual monitor, the TOO was a bit easier to detect than in this printed display. 
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The question for TOOs was always the same: “what weapons do you see and where are 
they located?” Location was to be specified in cardinal directions, thereby forcing a relatively 
high level of spatial-cognitive activity (e.g., Gugerty & Brooks, 2001). As with the CTs, these 
questions could only be answered once the operator had zoomed in close to the target. 

If the participant detected a CT or TOO, he or she was required to indicate detection by 
pulling the joystick trigger. The experimenter then pressed the appropriate button on the 
experimenter’s keypad to indicate whether this was a TOO or CT. After deciding that the UAV 
was close enough to the target to begin inspection, the participant pressed the loiter button (loiter 
would be selected either left or right) on the joystick (see Figure M3). This put the UAV into an 
automated oval pattern around the target. This oval pattern was 1.3 kilometers wide and 2.1 
kilometers long, and took between 2.5 to 3 minutes to complete an entire 4.8-kilometer circuit. 
The UAV turned 3 degrees per second at the ends of the oval. During the loiter pattern, the 
participant was able to use the x- and y-axes of the camera, as well as to zoom in and focus more 
closely on the target. The task of keeping the TOO in view while zooming and moving and 
keeping track of cardinal directions, was extremely challenging. 

After making the report, the participant could then depress the loiter button again, which 
would unloiter the UAV and unzoom the camera, returning the egocentric view to 6000 feet 
altitude. In the Baseline and Auditory conditions, once the report was completed, the participant 
had to relocate CT coordinates and reorient the UAV to the direct path to the CT. In the 
Automation condition, the UAV automatically resumed the correct path to the next CT. The 
duration of time between detection and completion of the final report was recorded as the 
response time measure. There was no separate measure of detection time (i.e., the time between 


27 



the appearance of the TOO on the screen, and the pilot’s depression of the “detect” button), since 
the fonner event was difficult to establish on a case by case basis, given variability of the course, 
and the camera angle. 

System Monitoring Task 

During each mission, participants were also asked to detect system failures (SF). A SF 
occurred when a system gauge needle went out of bounds (i.e., passed from the green zone into 
the red zone at either the top or bottom of the gauge; see Figure Ml). Each SF lasted 
approximately 30 seconds before automatically resetting (i.e., moving back into the green zone) 
if not detected. Not every leg contained a SF, and no leg contained more than one SF, although 
participants were not told this. The number of correct detections and the time it took to detect the 
SFs were recorded. 

If a SF was detected, the participant pressed a “detect” button on the joystick. Then he or 
she pressed the appropriate button on the keypad to indicate which system had failed. Lastly, the 
pilot typed in current ownship coordinates and then hit Enter. The duration of this time between 
detection and final report completion was recorded. 

The SFs were categorized under 4 types: A) during initiation of flight heading (i.e., while 
the pilot was consulting the message box and the 2D map, deciding how to turn the plane, and 
establishing the correct course); B) during regular flight, when no TOO was visible; D) 
approximately 5 seconds after a TOO loiter pattern was entered (i.e., the TOO had been detected 
and image inspection had begun); E) approximately 5 seconds after a CT loiter was entered. 
These will be referred to below as SF A , SF B , SF D , and SF E . Figure M8 shows a typical mission 
leg and the locations where SFs might occur along the leg. 



Orientation DuringT00 During CT 

© 0 © © © 


Figure M8. A timeline of SFs for a typical mission leg. 


Procedure 

Each participant was seated in a comfortable chair in front of the mission monitor. After 
signing the consent fonn, participants were asked to read the instructions for the experiment. 
Once they completed the instructions, they were allowed to spend 10-12 minutes on a practice 
mission, during which they would be exposed to two CTs, two SFs, and one TOO. Any questions 
they might have were answered by the experimenter. Once the practice mission was completed, 
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the participants were asked if they felt comfortable with the controls and instructions. All the 
participants responded positively and none of them asked for more time. 

The experimenter then started the first mission. Immediately, the instructions (i.e., “fly 
to” coordinates and CT question) for the first mission leg were posted visually (for Baseline and 
Automation conditions) in the Message Box, or aurally (for the Auditory condition). Throughout 
each mission, the participants were free to choose their own strategies for target search and 
systems monitoring. They were allowed to complete each task in the order they chose. While 
they were instructed to give equal priority to all tasks, at no time during the missions were they 
critiqued if they decided to give one task more priority than another. During dual-task situations, 
pilots were able to choose their own method of handling concurrent tasks; that is, they were free 
to serially process or parallel process as they saw fit. 

Design 


A mixed-subjects design was employed, in which all 36 participants were exposed to one 
of the three conditions in both a single- and dual-UAV scenario. The number of UAVs and 
mission maps were counterbalanced. Since there were three different maps (A, B, & C), defined 
by their legs, target locations, and target types, these were crossed with display condition and 
number of UAVs. 

Dependent variables included: 1) tracking error; 2) detection rates, response times, and 
accuracy for SFs and TOOs; 3) response times and accuracy for CTs; and 4) repeats (i.e., 
requests for repeated viewing or hearing of the CT instructions). 

To review, there are four human information processing tasks for which the pilot is 
responsible; 1) tracking, 2) monitoring for TOOs, 3) monitoring for SFs, and 4) memory task (of 
the CT location and instructions). These four tasks can be collapsed into three major goal- 
oriented tasks: 1) navigation task, 2) TOO task, and 3) SF task. The three goal-oriented tasks are 
shown in Table Ml. Each task is broken down into a series of subtasks, that typically appear in 
sequence. 
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Table Ml. Task analysis. 

1. Navigation Task: 

1.1 Read (or hear) CT location 

1.2 Establish coordinates (by orienting vehicle by joystick control or typing) 

1.3 Monitor heading toward CT location on 2D map (and re-orient if necessary) 

1.4 Refresh memory for location and final report 

1.5 Inspect image 

1.5.1 Enter loiter 

1.5.2 Zoom in 

1.5.3 Adjust camera orientation 

1.5.4 Count identify and/or assess cardinal orientations 

1.5.5 Verbal report of content. 

2. TOO task: 

2.1 Monitor 3D display 

2.2 Inspect image if target located 

(see 1.5 for sub tasks) 

3. System Failure Task: 

3.1 Monitor for System failures 

3.2 Identify failure 

3.3 Keyboard data entry 

Results 

This results section has been divided into six main subsections. The first three 
subsections correspond to the three goal-oriented tasks discussed in the Methods section, while 
the last three subsections deal with inter-workstation timelines, subjective pilot ratings, and 
IMPRINT modeling. The Discussion section will be used to establish and explain the 
relationship between the various subsections and how they contribute to model testing and 
validation. 

For the most part, mixed-design statistics were employed to analyze the data; however, 
due to missing data points in some perfonnance measures (e.g., if a target is not detected, then 
the corresponding SF will never occur), a between-subjects analysis was conducted to save the 
remaining data. The between-subjects analysis is generally a more conservative approach. 

In general, our interests lay in analyzing the effects of the two offload conditions 
(baseline and automation) when compared to the baseline condition. These two planned 
comparisons were developed prior to running subjects, so there were no adjustments made (e.g., 
Bonferroni) to control familywise Type 1 error rates (see Keppel, 1982, for a more detailed 
explanation of this approach). Furthermore, one-tailed t-tests were frequently used because we 
expected improvements due to the two offloads. 

Because the current experiment included a single-UAV flight control, the results often 
replicated previous findings (see Wickens & Dixon, 2002). However, because of the added 
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incentives and subsequent increased effort, there are differences between the experiments that 
are highlighted in this section. 

Lastly, the current study originally included a SFc, which was designed to occur just 
before a TOO appeared on the 3D display; however, due to programming issues, these SFs did 
not measure precisely what they were intended to measure. Because they were still legitimate 
SFs, the data were included in overall ANOVAs, but because of their temporal uncertainty, they 
were not included in more detailed comparisons between conditions and SF types. 

In the first three sections, we analyze the main objective performance measures: 1) 
mission completion, 2) target of opportunity (TOO) analysis, and 3) system failures (SF). Table 
1 presents an overview of the data from all tasks across both single- versus dual-UAV flight 
control and the three flight conditions. 


Table 1. Performance data for all major tasks. 


Single 


Dual 

Baseline Auditory Automation 


Baseline Auditory Automation 


Tracking Error (RMS - meters) 

2785 

2775 

CT Response Time (secs) 

28.77 

21.85 

CT Accuracy (%) 

86 

86 

Number of Repeats (per leg) 

2.90 

1.78 

TOO Detection Rate (%) 

57 

45 

TOO Report Time (secs) 

18.84 

19.87 

TOO Report Accuracy (%) 

79 

75 

SF Detection Rate (%) 

85 

92 

SF Detection Time (secs) 

7.16 

3.58 

SF Correction Time (secs) 

11.40 

9.44 

SF Report Accuracy (%) 

90 

93 


1.0 Mission Completion 


0.00 

2796 

2766 

0.00 

21.23 

29.06 

26.91 

25.58 

89 

92 

78 

88 

1.50 

41.63 

31.25 

18.88 

92 

40 

28 

79 

18.85 

22.32 

20.97 

17.42 

89 

77 

71 

90 

93 

73 

98 

83 

8.31 

10.91 

3.91 

11.19 

11.80 

10.72 

9.76 

11.09 

93 

90 

97 

88 


As in the previous study, tracking again benefited from automation (zero error). A mixed- 
design analysis of variance revealed that tracking did not benefit from the auditory offload [F(l, 
22) < 1.0]. Furthermore, tracking error in the dual-UAV scenario was essentially equivalent to 
single-UAV flight across both baseline and auditory conditions [F(l, 22) < 1.0]. This suggests 
that pilots placed primary importance on the tracking task and allocated enough resources to that 
task to ensure its successful completion regardless of conditions or number of UAVs. This 
resource allocation policy presumably eliminated any benefit provided by offloading concurrent 
task to the auditory channel. 

Response times to command targets (CT) were essentially equivalent across conditions 
[F(2, 33) = 1.09, p = .35], and number of UAVs [F(l, 33) = 2.2, p = .15], with no interaction 
between load and condition [F(2, 33) < 1.0]. However, the raw data suggests a trend towards 
larger report times in dual-UAV flight control, when some attention may need to be diverted to 
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monitoring the other UAV. CT report accuracy followed the same pattern [F(2, 33) = 1.86, p = 
.17; F(l, 33) < 1.0], suggesting that neither offload nor number of UAVs had any effect on 
completion of the report task. Again, pilots appeared to treat this task of fulfilling the mission 
command objectives as primary, safeguarding it against workload demands. 

During each mission leg, pilots were required to read and memorize fly-to coordinates 
and report questions. If necessary, they were able to refresh their memory by pressing a repeat 
button, which would trigger another 15-second presentation of the flight instructions (played one 
time in the auditory condition). Figure R1 presents the number of repeats summed over the 10 
legs during each of the six possible conditions. 


Average number of Repeats per leg 



Condition 


Figure Rl. Average number of repeats per leg across conditions and number of UAVs. 


As in Experiment 1, the auditory condition led to fewer repeats when compared to the 
baseline condition [F(l, 22) = 7.78, p = .01], regardless of the number of UAVs, as seen in the 
lack of an interaction effect [F(l, 22) < 1.0]. This was presumably because pilots were able to 
offload some of the visual demands to the auditory channel. In the baseline condition, pilots had 
to divide foveal attention between all the different windows. Because of the wide visual angles 
between these areas of interest (7 - 23.5 degrees on one UAV workstation), parallel processing is 
effectively prevented. By relieving the pilot of having to read flight instructions while 
monitoring other visual displays, visual scanning became less likely to create interference, and 
resulted in improved performance, a benefit that was equally realized in both single and dual 
UAV conditions. 

Figure R2 presents a timeline of repeats for a sample mission leg. Each mission leg is 
divided into sections representing a particular percentage of leg completion. Most evident from 
the figure is the frequent use of the repeat button early in the leg, as pilots are presumably 
assuring their correct identification of the CT coordinates and path to be flown, and again later in 
the leg, as pilots are refreshing the report question for the CT. The figure also reveals that the 
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auditory condition appears to improve performance throughout the mission leg, particularly in 
single-UAV flight control. In dual-UAV flight control, the benefit is more apparent at the 
beginning and end of each leg, indicating less need to initially refresh the fly-to coordinates and 
also less need to refresh the report question just prior to entering a CT loiter. 


Number of Repeats across all conditions 



—■—Baseline 
(single) 

■ - Baseline 

(double) 

—*— Auditory 
(single) 

■ ■ ■ a- ■ Auditory 

(double) 

—♦— Automation 
(single) 
Automation 
(double) 


Figure R2. A timeline of repeats for a sample mission leg. Negative values indicate repeats 
during periods of time when the pilot is initiating heading in the wrong direction, and values over 
100% indicate CT overshoots. 


Automation also led to fewer overall repeats [F(l, 22) = 27.28, p < .001] than the 
baseline condition, but for different reasons than the auditory offload. As seen in Figure R2, the 
reduction in repeats was most apparent in the middle of the mission legs, indicating that pilots 
felt no need to refresh fly-to coordinates since they knew the auto-pilot would guide them 
correctly to the next CT. However, a spike in the number of repeats both during the initial 
heading and just prior to a CT loiter indicates that pilots did not benefit as much from automation 
when they were trying to input fly-to coordinates or recall report questions, although there is 
some improvement over the baseline condition. 

As both figures show, the number of repeats increased dramatically during dual-UAV 
flight when compared to single-UAV flight [F(l, 33) = 43.18, p < .01]. This dual-UAV 
decrement is not as severe in the automation condition as in the other two conditions, as reflected 
by the condition x load interaction [F(2, 33) = 4.08, p < .05], a difference which can be explained 
by the lack of need to refresh fly-to coordinates during automated flight control regardless of the 
number of UAVs. 

2.0 TOO monitoring 

Figure R3 presents the percentage of TOOs detected on average throughout each mission. 
A mixed-design analysis of variance indicates a strong main effect for condition [F(2, 33) = 
18.29, p < .001] on TOO detection rates. However, this effect only appears to be manifest when 
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comparing baseline results to automation. A comparison between the baseline and auditory 
conditions [F(l, 22) = 1.47, p = .24] reveals that auditory offloading had no effect on TOO 
detection rates. 


TOO Detection Rate 



□ Single 
■ Dual 


Condition 


Figure R3. TOO detection rate across conditions and number of UAVs. 


By contrast, the automation condition facilitated a much greater TOO detection rate than 
the baseline condition [F(l, 22) = 24.96, p < .001], equally so in both the single- and dual-UAV 
flight controls, as evidenced by a lack of interaction between condition and number of UAVs 
[F(l,22)< 1.0]. 

As in the previous experiment, there appear to be two explanations for this effect. First, 
the auto-pilot, by nature, always guided the UAV in a direct, straight-line path to the next CT, 
thus ensuring that all TOOs would appear in the 3D display. In the baseline condition, tracking 
error sometimes prevented pilots from having the opportunity to detect TOOs. However, even 
when comparing results of TOOs that actually did appear in the 3D display during both 
conditions, there is still an improvement in TOO detection rates for the automation condition. In 
the single-UAV flight control, of the 73% of TOOs that came into the 3D display during the 
baseline condition, only 76% were detected, as compared to 92% for the automation condition 
[marginally significant gain of 16%: t(l 1) = 1.87, p < .10]. In the dual-UAV flight control, of the 
68% of TOOs that came into the 3D display during the baseline condition, only 52% were 
detected, as compared to 79% for the automation condition: a significant gain of 27% [t(l 1) = 
3.31,p < .01]. This suggests that a second reason for improved detection must exist; that is, the 
auto-pilot was able to allow reallocation of perceptual, cognitive, and motor resources, originally 
used for manual tracking, to the TOO detection task. 

TOO detection rates dropped about 10-20% when adding a second UAV workstation, 

[F( 1, 33) = 20.01, p < .001], with decrements that were relatively equal across all three 
conditions, [F(2, 33) < 1.0]. This suggests that increased visual angles between the tasks resulted 
in decreased ability to foveate on the necessary displays. Furthermore, as seen below in the 
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timeline analysis, TOOs sometimes occurred simultaneously on both workstations, increasing the 
difficulty level of detection. 

Figure R4 presents the TOO report times across conditions and number of UAVs. An 
overall analysis of variance reveals no statistically significant differences between condition or 
number of UAVs for TOO response times. 


TOO Report Times 



□ Single 
■ Dual 


Condition 


Figure R4. TOO report times across conditions and number of UAVs. 


While the single-UAV results do appear equivalent across conditions, a dual-UAV post 
hoc comparison between the baseline and automation conditions reveals a marginally significant 
difference in response times [t(21) = 1.86, p < .05], with an almost 5-second advantage for the 
automation condition. This can be explained by the simple fact that pilots did not need to 
monitor the tracking task in one workstation while dealing with TOOs in the other workstation 
during the automation condition. In contrast, visual and motor demands of tracking in the 
baseline condition would almost certainly interfere with visual and motor demands in the TOO 
inspection and report task. 

With regards to TOO report accuracy, a mixed-design analysis of variance revealed no 
main effects for either condition [F(2, 33) = 1.30, p = .29] or number of UAVs [F(l, 33) < 1.0]. 
This is consistent with the report accuracy results found for CTs. 

3.0 System Failures 

In the previous experiment, the auditory condition was found to improve SF detection 
rates and facilitate shorter response times for SFs which occurred during relatively routine 
concurrent tasks (i.e., SF A and SF B ), but not during SFs which occurred in conjunction with 
highly challenging tasks (i.e., SF D and SF E ). The assumption was made that pilots were forced 
into “cognitive tunneling” because of the difficulty of the target inspection task. To overcome 
this single channel tunneling, pilots in the present experiment were rewarded with monetary 
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prizes for exceptional performance. It was hypothesized that increased rewards would lead to 
equal increased effort, thus resulting in improved parallel processing during the auditory 
condition. Figure R5 presents SF detection rates for the six conditions. 



Figure R5. Overall SF detection rates across conditions and number of UAVs. 


A mixed-design analysis of variance between the baseline and auditory conditions shows 
a main effect for condition [F(l, 22) = 5.29, p = .03], but not for number of UAVs [F(l, 22) < 
1.0]. The lack of interaction between condition and number of UAVs [F(l, 22) = 1.70, p = .21] 
suggests equally improved detection rates in both the single- and dual-UAV flight controls when 
the SFs were offloaded to the auditory channel. Further statistical analyses between specific SF 
types were unrealistic due to the paucity of individual data points and the resulting lack of 
statistical power. However, Figure R6a presents this breakdown of the SF types in order to show 
patterns in the data which suggest that the auditory condition did not suffer across SF type as 
seen in Experiment 1. In fact, detection rates in the auditory condition approached perfect 
performance for almost all of the SF types. Most importantly, the decrement in detection of SF D 
and SF e , which in the previous experiment were seen as symptoms of cognitive tunneling, were 
not observed here. This pattern is almost certainly not the case for the baseline condition, which 
fell well below 90% in most cases, particularly in the dual-UAV flight control. 
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Figure R6a. SF detection rates across SF types, conditions, and number of UAVs. 


A mixed-design analysis of variance between the baseline and automation conditions 
shows no main effect for condition [F(l, 22) = 1.38, p = .26], and a marginally significant effect 
for number of UAVs [F(l, 22) = 3.28, p = .08]. Further paired comparisons between the single- 
and dual-UAV flight controls in the baseline condition [t( 11) < 1.0], and the automation 
condition [t(l 1) = 2.20, p < .05], show this dual-UAV decrement to be present only in the 
automation condition. Figure R7 reveals that this dual-UAV decrement only appears to show up 
in two of the four SF types for the automation condition. 
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Figure R6b. SF detection rates across SF types, conditions, and number of UAVs. 
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In contrast to the previous experiment, SF response times in the current experiment were 
divided into two subsections: detection times and correction times. Detection times indicate how 
long it took the pilot to notice the SF and press the detection button, while correction times 
indicate how long it took to actually diagnose and correct the SF and input the current UAV 
coordinates. 

Figure R7 presents the overall SF detection times collapsed across SF type. A between- 
subjects analysis of variance for detection times revealed an overall main effect of condition 
[F(l, 294) = 35.54, p < .001]; however, planned comparisons between the baseline and offload 
conditions showed this effect to only be present when compared to the auditory condition [F(l, 
188) = 54.37, p < .001], and not to the automation condition [F(l, 195) = 1.06, p = .30]. 



Figure R7. Overall detection times across conditions and number of UAVs. 


Figure R7 reveals that the auditory condition facilitated detection times that were less 
than half as long as the baseline condition. Furthermore, the dual-UAV decrement that once 
again shows up in the baseline [F(l, 89) = 4.05, p < .01] and automation [F(l, 106) = 5.42, p < 
.05] conditions is not present in the auditory condition [F(l, 99) = 1.66, p = .20]. This finding is 
consistent with the detection rate data which shows that the auditory condition does not suffer 
when adding an addition UAV workstation. 

Figure R8 presents detection times broken down according to SF type. Two important 
features immediately arise from this graph. First, as reflected by the two lines at the bottom of 
the graph, not only does the auditory condition support overall equal performance between 
single- and dual-UAV conditions, but this trend appears to be consistent across all SF types [p > 
.05 for all single-dual comparisons]. The results show this is not always the case for the baseline 
[SF a : p < .05; SF B : p < .05] or automation [SF B : p < .01] conditions. 
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Figure R8. SF detection times across SF types, conditions, and number of UAVs. 


Secondly, while there is a strong statistical effect of SF type [F(4, 99) = 3.80, p < .01] in 
the auditory condition, due to very low variance, this effect is not practically strong. The largest 
difference between SF types appears to be only 2-3 seconds, while the baseline [F(4, 89) = 4.16, 
p < .01] and automation [F(4, 105) = 3.70, p < .01] conditions reveal much larger 3-11 second 
differences. This suggests that the cognitive tunneling effect found in the prior experiment is not 
nearly as strong in the current experiment for the auditory condition; that is, when combining a 
SF task with the highly difficult task of target inspection (i.e., SF D and SF E ), performance in SF 
detection does not appear to suffer when compared to a SF combined with an easier concurrent 
task (i.e., SF A and SFb). This lends credence to the theory that cognitive tunneling may be more 
of a strategic effect rather than a structural one. In other words, pilots tend to process serially out 
of choice, and not out of necessity. By offering incentives, as we did in the current experiment, 
this strategic tunneling can be overcome. 

SF correction times were measured by how long it took to correct and report the location 
of the SF. A between-subjects analysis of variance revealed no main effects for condition when 
comparing the baseline condition to the auditory [F(l, 189) = 1.98, p = .16] or the automation 
[F(l, 196) < 1.0]. These results are expected since all conditions are essentially equal once the 
SF has been detected. Although there might have been a slight reduction in workload in the dual- 
UAV automation condition because pilots were not required to manually fly the UAV, this 
advantage would have been very small indeed because once the initial heading had been set, the 
UAV flew a straight-line course free of turbulence. Because of this, pilots did not need to 
concern themselves with checking the flight path of the UAV during the short window of time it 
took to diagnose and correct a SF. 

4.0 Timeline Analysis 

The purpose of the timeline analysis in the current study is to examine the effects that 
performing tasks on one UAV workstation has on tasks in the other workstation. In this section, 
the focus of analysis was to determine exactly when a certain type of task was occurring on both 
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workstations simultaneously, and then to compare how each of the offloads might have improved 
this dual-task performance. The tasks that were compared included: SFs, TOOs, and CTs on both 
workstations. These were paired up in the following ways: TOO + SF, TOO + TOO, TOO + CT, 
CT + SF, CT + TOO, CT + CT. The pairs were analyzed for both left and right workstations, 
resulting in a total of 36 combinations (6 pairs x 2 workstations x 3 conditions). Figure R9 
presents the overall data collapsed across all pair, workstations, and conditions. 


Overall (cross-monitor interference) 
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Figure R9. Overall response times to pairs of tasks. 


In Figure R9, the “serial” value indicates how long the response should be if single 
channel serial processing took place; that is, the sum of the two average task times for that pilot, 
as each task time was computed when the task was performed without a task (other than 
monitoring) done on the other workstation. The “ideal” value indicates how long it should take if 
perfect parallel processing took place; that is, if both tasks were done simultaneously with no 
task interference. The “actual” value is how long it actually took the pilot to perfonn both tasks 
in their entirety. 

From the graph, it is apparent that the actual times for most tasks was greater than the 
sum of the two average task times, thus suggesting a pattern of switching costs between tasks. 
Pilots were apparently trying to switch back and forth between tasks, costing them additional 
time, when it might have been more efficient to simply complete one task before beginning the 
other. It is also evident from the graph that this cost was much greater in the baseline condition 
than in the auditory [t(25) = 3.01, p < .01] or automation [t( 30) = 4.34, p < .001] conditions. 

This effect is not surprising, since automation reduces the need to monitor the flight path and the 
auditory offload reduces the need to monitor the SF display. 

It should be noted, however, that these results are diluted somewhat by including 
combinations of tasks that do not inherently benefit the auditory condition. For example, TOO + 
TOO combinations are probably not as likely to improve in the auditory condition as TOO + SF 
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combinations. Figures RIO and R11 present data separately for TOO + SF and CT + SF 
combinations, respectively. 



Figure RIO. Response times to TOO + SF combinations across workstations. 


CT + SF (cross-monitor interference) 




Figure R11. Response times to CT + SF combinations across workstations. 


The results from Figures RIO and R11 show a more salient improvement in the auditory 
condition over the baseline condition with respect to parallel processing. While the paucity of 
data points prevents a statistical comparison, it is obvious that during the few opportunities that 
pilots had to dual-task (target inspection plus SF) across workstations, they were able to parallel 
process when offloading the SF task to the auditory channel. This is suggested because in the 
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auditory condition, the “actual” performance is better than the “serial” performance. In contrast, 
pilots performed 2-3 times worse than ideal in the baseline condition. This finding is expected, 
given that the visual angles between these tasks (the two display workstations) was over 30 
degrees, which, in most cases, requires head movement to switch between tasks. 

The automation condition also facilitated improved performance across workstations 
when compared to the baseline condition, suggesting a similar pattern with data in the same 
workstation. Time (SCT) and mental resources (SRT and MRT) saved by not having to manually 
track in the automated condition were reallocated to the task combinations discussed here. The 
latter data provide an interesting convergence with the findings of Liao and Moray (1993), 
whose modeling efforts revealed evidence for parallel processing with two tasks, but not with 
four. Although the mapping between their tasks and ours is far from perfect, the current data also 
suggest that when three active tasks were required (tracking on one workstation with SF and 
TOO inspection distributed across both workstations), parallel processing was disabled in the 
baseline condition. But when tracking was automated, and the tasks were thereby reduced to two, 
evidence for parallel processing reemerges. 

Another prediction of strict SCT, outlined in the introduction, was tested by assessing the 
delay by which a second-arriving task event arrived following the initiation of a first-arriving 
task, when the fonner arrived before the latter was completed. According to the SCT predictions, 
unless there is switching between tasks, every one-second shortening of this arrival delay will 
produce a concomitant one-second lengthening of response time to the second-arriving task 
(RT 2 ). In Figure R12, these values of RT 2 are plotted against the arrival delay for the total 
sample of data points in which there was task time overlap between the two tasks, one present on 
each workstation. 


Correlation between ISI and RT (across conditions) 
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Figure R12. Correlation between interstimulus interval (ISI) and RT to the secondary task. The 
data is collapsed across all conditions. 


The data in Figure R12 reveal no evidence for the negative correlation, with a slope of -1, 
that strict SCT would predict (r = 0.13), and so are inconsistent with a strict single channel 
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model, by which engagement of a cognitive process on one UAV workstation will totally “lock 
out” the initiation of performance on second UAV workstation until the former is complete. 
Similar results were obtained when breaking the data down into separate dual-task pairs (e.g., CT 
vs. SF; TOO vs. SF, etc.). The present analysis does not, however, reject all forms of single 
channel theory. For example, they could be consistent with a switching model in which attention 
is switched immediately to the later arriving task, and the ongoing task bears the cost of 
concurrence and delay. 

A subsequent analysis on RT to the first arriving task revealed that its performance time 
was actually shortened, rather than lengthened, as the inter arrival time decreased, showing a 
significant positive correlation of 0.68. We interpret this finding to mean that, if a task arrived on 
one work station shortly after the pilot initiated perfonnance on the other, the pilot accelerated 
performance on the latter task (perhaps by mobilizing more effort) to deal with the imposed dual 
task demands. 

5.0 Single Resource Theory Predictions. 

In Table 2 we presented the predicted demand value of various component tasks. 
According to a pure version of single resource theory, the interference between tasks should be a 
direct function of their task demand value. In order to test these predictions, the total concurrent 
demand value of each single and dual task combination was computed, simply by summing the 
demand values of concurrent tasks. There were 3 single and 3 dual task combinations. 

Predictions were generated for the following four dependent variables: # retrievals, TOO 
detection rate, SF detection rate and SF detection time. These DVs were chosen because, unlike 
tracking error, they were valid measures across all 6 conditions, and unlike some of the other 
measures, there were substantial significant differences between conditions, a circumstance 
meaning that there were meaningful differences in performance to be predicted. 

The correlations (N=6) between predicted and obtained data were 0.96 (# retrievals), 0.93 
(TOO detection rate), 0.49 (SF detection rate) and 0.18 (SF detection time). In interpreting these 
results, it is not surprising that the data for the two SF task measures reflect a poor model fit. A 
single resource model obviously does not account for differences between auditory and visual 
display of information, differences which, as noted in section 3 above, accounted for large 
amounts of variance in SF detection performance. On the other hand, predictions of the single 
resource model across the other two aspects of task perfonnance were remarkable good, 
accounting for over 85% of the variance of the data. 

6.0 IMPRINT Modeling 

Through the use of IMPRINT software, it is possible to simulate experimental data using 
three different task interference models: 1) VACP (visual-auditory-cognitive-psychomotor), 2) 
Goal-oriented, and 3) Advanced workload analysis. The latter of these analyses is based 
primarily on the Multiple Resource Theory, developed by Kantowitz and Knight (1976), Navon 
and Gopher (1979), and Wickens (1980, 1991). This is the analysis used in the current study. 

The first step in building a simulation with the IMPRINT software is to define the 
mission. Figure R13 shows a sample screen where the modeler can choose a time standard, time 
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criterion, accuracy criterion, and mission criterion. This determines how often the mission must 
meet, and successfully complete without abort, the time standard. This is also where the modeler 
must choose which type of task interference theory to represent the data. In the current study, we 
chose a time, accuracy, and mission criterion of 90%. 


[Mission Information 



Figure R13. A splash screen of mission information. 

The next step is to detennine the functions and tasks. Examples of functions might be 
“initiate heading” or “target analysis”. Tasks are more specific operations within each function. 
For example, when a pilot initiates heading, they might “check flight path coordinates”, “set 
course”, “adjust heading”, “manipulate joystick”, etc. Functions and tasks can either be defined 
in the screen labeled Figure R14, or later in the Network Diagram. 
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Function and Task List 




Network Diagram 


Mission: 

Function: 


UAV Baseline 


FunctionID 

Function Name 

▲ 

1 

Initiate Heading 


2 

Initiate Heading (with SFa) 


3 

Tracking 


4 

Tracking (with SFb) 


5 

Tracking (with SFc) 


6 

TOO Analysis 


7 

TOO Analysis (with SFd) 


8 

CT Analysis 
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11 


TaskID llaskTt ame 


List Sub-Nodes 


Go To Parent 
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Paste Node 


□K 


.Cancel 


Help 


Figure R14. Function and Task list. Clicking on the “list sub-nodes” button will display tasks 
associated with each function. 
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In the current study, we defined 9 separate functions, with tasks listed below each 
function: 

1. Initiate heading 

a. Set course 

b. Monitor SF 

2. Initiate heading with SFa 

a. Set course 

b. Correct SF A 

3. Tracking 

a. Continue course 

b. Search for targets 

c. Monitor SF gauges 

4. Tracking (with recall) 

a. Continue course 

b. Search for targets 

c. Monitor SF gauges 

d. Recall coordinates and report question 

5. Tracking (with SF B ) 

a. Continue course 

b. Search for TOO 

c. Correct SF B 

6. TOO analysis 

a. Analyze TOO 

b. Monitor SF gauges 

7. TOO analysis with SF D 

a. Analyze TOO 

b. Correct SF D 

8. CT analysis 

a. Analyze CT 

b. Monitor SF gauges 

9. CT analysis with SF E 

a. Analyze CT 

b. Correct SF E 

For purposes of simplicity, the current study did not attempt to parse the tasks into minute 
items of interest; rather, each task represents an assignment that may consist of many smaller 
sub-tasks. For example, one task might be to “search for targets”. Theoretically, this could be 
broken down into smaller sub-tasks, such as “scan 3D display”, “analyze terrain patterns”, etc., 
but for our purposes, we chose to limit the model to tasks that had a clear purpose, with a 
beginning and an end. 

Next, the modeler can open the Network Diagram and detennine where to place each 
function and task. The network diagram, shown in Figure R15, also allows the modeler to 
determine specific information about each task. 
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Figure R15. A sample network diagram showing mission functions. 


If a modeler wants to add tasks to each function, then she can open that function and 
build a new diagram of tasks, which are affixed to that function. Figure R16 shows a sample task 
diagram. The modeler must state here the order of tasks and how they are to be performed 
temporally. If several tasks are to be completed simultaneously, as done with each function in the 
current study, then multiple paths go through each task and link back together again after 
completing all three tasks. The diamond with the ‘M’ in it, located just right of the START task, 
represents a multiple task branching logic, which we used as part of the Advanced Workload 
Analysis. 
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Figure R16. Task diagram. Clicking on a task opens a Task Information screen. 



Figure R17. Task Information screen. 
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Figure R17 shows where many of the details regarding each task can be manipulated. 
Here is where the modeler detennines the time and accuracy required of each task, the possible 
effects of multiple tasks, how to treat task failures, the workload measures associated with tasks 
(not used in the Advanced Workload model), crew assignment for each task, and additional 
taxons (not used in the Advanced Workload model). For our purposes, all accuracy and time 
standards were kept at 90%. This means that the operator had to complete the task in less than 1 
minute at least 90% of the time, and he also had to complete the task accurately at least 90% of 
the time. The crew assignment was always the primary operator. When a task failed to complete 
due to time lapse, usually caused by resource overloads, the task was repeated until completed 
successfully. 

The next step is to define the workload and crew member parameters, which involves 
several steps and is based on Multiple Resource Theory. First, the modeler must define the 
resources and interfaces. In the current study, the resources were visual, auditory, motor, speech, 
and cognitive resources. These were matched with the following interfaces: keypad, message 
box, 3D display, 2D top-down display, SF gauges, joystick, and the type of response pilots used. 
Table R2 shows how we paired the resources to the interfaces. An “X” represents a combination 
between a particular resource and the interface it is aligned with. 


Table R2. Paired resources and interfaces. 



Visual 

Auditory 

Cognitive 

Motor 

Speech 

Keypad 

X 



X 


MessageBox 

X 

X 

(Auditory 
condition only) 

X 



3D Display 

X 


X 



2D Display 

X 


X 



SF gauges 

X 

X 

(Auditory 
condition only) 

X 



Joystick 

X 



X 


Verbal 

Response 





X 


After defining the resource/interface (R/I) combinations, the modeler must assign R/I 
combinations to each mission task. For example, in the current study, the task “set course”, 
involved the following R/I combinations: Visual-MessageBox, Visual-2DTopDown, Motor- 
Joystick, Cognitive-MessageBox, and Cognitive-2DTopDown. 
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Next, the modeler assigns task demand values to each task. These task demands values 
can either be chosen from a list provided by IMPRINT, which is based primarily on previous 
literature that quantifies the difficulty of particular tasks, or they can be added arbitrarily by the 
modeler. In our simulation, we always accepted IMPRINT’S suggestions unless there was no 
clear match between the suggestion list and the actual task we modeled. In those cases, we 
interpolated the value by using the two closest values on the list. 

Lastly, the modeler assigns the channel conflict values, based on the conflict matrix 
developed from Multiple Resource Theory. 

6.1 IMPRINT results 

It should be noted that the parameters used in the IMPRINT modeling were based on 
Wickens (2002) recommended values, and were not adjusted post hoc. Figure R18 presents the 
predicted interference values generated by IMPRINT for the three conditions across each of the 9 
dual-task situations. 


Operator Workload Table (IMPRINT) 


a> 



Figure R18. Interference values for dual-task combinations. 


The results from Figure R18 clearly show strong MRT predictions of improved dual-task 
performance in the auditory condition over the baseline condition for monitoring / correcting SFs 
and recalling flight instructions, based on reduced interference between these task combinations. 
As seen in the graph, interference levels were lower for all 9 task combinations in the auditory 
condition. This correlates well with the actual performance data, which show improved SF 
detection rates (tasks 1, 3, 6, and 8 in the IMPRINT analysis) and detection times (tasks 2, 5, 7, 
and 9), as well as improved recall of the flight instructions (task 4) in the auditory condition. 
Furthermore, the level of improvement also seems to be fairly well predicted in the IMPRINT 
simulation; that is, the interference values generated by IMPRINT are 10-25% lower in the 
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auditory condition, which appears similar to the degree of improvement in most of the 
performance data. 

One noticeable difference is in SF detection times, which were more than twice as fast in 
the auditory condition, despite only a 10% reduction in interference values. This may be due to 
the nature of the task; detection time of a SF is simply a reaction time task, and a slight reduction 
in dual-task interference may be all that is necessary to drop below the threshold of cognitive 
tunneling on another task. Once the mental workload has dropped below this arbitrary threshold, 
then parallel processing is feasible, and the pilot is able to respond instantly to the SF rather than 
wait until the side task is finished (i.e., serial processing). 

With regards to TOO detection, the IMPRINT results appear to predict improved 
performance in the auditory condition, but failure to see these results in the performance data is 
probably due to pilot choice. Pilots were free to allocate extra resources, freed up by auditory 
offload of one task, to either of the two tasks in a dual-task situation. It is apparent from the data 
that they chose to allocate these resources to the SF task and not to the concurrent TOO task. 

One weakness in IMPRINT modeling is that it cannot predict where the operator will allocate 
these resources. 

In the automation condition, IMPRINT correctly predicted improved tracking and 
detection rates for some SFs (i.e., SF A and SF B ), but incorrectly predicted faster detection times 
to these SFs. Theoretically, it seems logical that pilots should have been able to response more 
quickly to SFs since they had extra resources available, but the data do not show this. As 
mentioned earlier, one explanation may be that the tracking task in the baseline condition was so 
easy the difficulty level between the two conditions was too similar to facilitate improved 
performance in the automation condition. A post hoc adjustment of IMPRINT task demand 
values would probably correct this discrepancy. 

The IMPRINT simulation did an exceptional job of predicting TOO detection rates in the 
automation condition. Because of the reduced task interference between tracking and target 
search, pilots were indeed able to detect almost twice as many TOOs in the automation condition 
than in the baseline condition. This corresponds with the 50% reduction in task interference, 
although it should be noted that levels of task interference are not necessarily expected to 
correlate serially with levels of actual performance. 

In summary, IMPRINT appears to generate simulation results that juxtapose quite nicely 
with experimental data in the UAV environment. The foundation of the IMPRINT model used in 
the current study is based on MRT, and helps to validate that theory’s model of task interference, 
particularly when the operator divides tasks between two different modalities. 
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Discussion 


Single task effects 

The general pattern of results from the single task conditions of the present experiment 
appear to generally replicate the findings of the previous experiment (Wickens & Dixon, 2002). 
Thus, the automation offload generally improved TOO monitoring and SF detection, while the 
auditory offload assisted SF detection, but not the TOO monitoring task. As before, there 
appeared to be a large degree of interference in the baseline condition between monitoring the 
flight path, the TOO display, and the SF display, particularly in the added cognitive/motor 
requirements of “dealing with” command targets, targets of opportunity, or system failures (i.e., 
image inspection and report). One important contrast of the current results with those of the 
previous experiment is that here we found that the auditory offload did help SF detection, even 
when those system failures occurred during the cognitively demanding phases of TOO and 
command target image inspection. Thus in this experiment, unlike the previous one, the severe 
case of “cognitive tunneling” was not observed, a difference we attribute to the added incentives 
that were provided for good perfonnance. In the following discussion, we now turn to the joint 
effects of task load (one versus two workstations) and the two different offloads. 

Task loading effects 

The dual-UAV requirements imposed a fairly consistent decrement across most aspects 
of performance except the primary tasks of tracking (flight trajectory control) and image 
reporting accuracy. This “protection” of these primary task measures can be readily interpreted, 
given both its importance for the mission, and the fact that the flight control task itself was of 
sufficiently low bandwidth that pilots could readily divert attention from monitoring its track on 
one UAV to reallocate to other tasks in the other, without producing a deviation in performance. 
In contrast, both of the tasks that might be viewed as “secondary” (TOO monitoring, SF 
monitoring), as well as the report retrieval (memory) task, all showed some indications of a cost 
when the UAV supervision requirement was doubled. These costs were mitigated by the two 
different offloads in different ways, as we describe below. 

Auditory offload. The auditory offload had no observable benefit on tracking or TOO 
monitoring. We assume that the short “check glances” to the SF display, which were needed in 
the two visual conditions (baseline and automation), but avoided in the auditory condition, were 
of sufficiently short duration that the elimination of these short time spent away from the other 
tasks could not be used to improve their performance. This is in the same way that a good driver 
can make short glances downward to the instrument panel without sacrificing performance on a 
low bandwidth driving task (Horrey & Wickens, 2002). Associated with the primary task of 
mission goals, there was an auditory offload “benefit” to the CT information retrieval task, in 
that providing this information auditorially reduced the frequency with which such repeats were 
required. We assume that in the two visual conditions, pilots who were engaged in the tasks of 
tracking, TOO monitoring, and system monitoring, were often required to divert their gaze from 
the instruction box to monitor these channels, thereby degrading the quality of comprehension of 
this instructional material and forcing it to be re-read. Such diversion did not need to occur when 
the instructions were presented auditorially, given the availability of separate perceptual (visual 
versus auditory) resources. 
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Most notably, the auditory offload had a dramatic benefit for the SF task, to which the 
auditory offload was directly attached, as shown in Figure R8. The offload appeared to “buffer” 
this task, not only from the effects of dual-UAV loading, but also from the effects of cognitive 
tunneling, which had been previously observed by Wickens and Dixon (2002) during the target 
examination phase, for both the TOOs and the CTs. It is important to emphasize that, while there 
was a gain in performance for the SF task by its auditory delivery, such delivery did not impose 
any cost on the ongoing visual tasks, an effect that has sometimes been observed as a sort of 
“pre-emption” imposed by auditory onsets (Helleberg & Wickens, in press). 

Automation Offload. In contrast to the auditory benefit, primarily localized to the SF 
task, the benefits of automation were generally more widespread, an effect not altogether 
surprising because automation entirely eliminates a task (monitoring and correcting flight path 
trajectory), whereas auditory offload only altered the delivery of task-relevant information. 
Naturally, the benefit of automation to tracking perfonnance was an intentional artifact, as is the 
case with most autopilot studies (that is, the autopilot tracker is designed to produce nearly 
perfect perfonnance). In contrast to the auditory offload, automation showed a substantial benefit 
to the perceptually challenging TOO monitoring task, a benefit that was only partially attributed 
to the more accurate flight paths flown by the autopilot. That is, not having to monitor the 
tracking navigation display freed up visual resources to inspect the TOO image display more 
closely, and these released resources were even more advantageous in dual-UAV than in single- 
UAV conditions. That is, the automation benefit to the detection of targets that did pass through 
the TOO image display was both larger in percentage, and more significant statistically, in dual- 
than in single-task conditions. Furthermore, in dual-, but not single-task conditions, this benefit 
was reflected in the slightly shortened the time required to zoom in and report a TOO identity. 

The automation offload also assisted performance on SF monitoring, improving its 
detection accuracy equally in both single- and dual-UAV conditions, although it had no benefit 
on SF detection time. It should be noted here that in both the automation and the baseline 
conditions, the need to monitor a second UAV slowed the SF response time by about 3 seconds 
(see Figure R7). One way of interpreting this value is that it reflects the frequency with which 
attention is switched between the two UAV workstations, at approximately 3-second intervals. 
We revisit this issue below, in considering the relevance of single channel theory. 

Theoretical Interpretations and relevance for workload models. 

The current data provide evidence for some single channel aspects of behavior, 
particularly within a single-UAV. We assume, given the high acuity demands of the TOOs, that 
pilots simply could not monitor this display for their appearance when their visual attention was 
occupied elsewhere, either within the same workstation, or on the other workstation. Also, as 
noted above, we assume that the 3-second dual task cost to visual SF detection described above 
probably reflected the inability to see a SF on one UAV, when vision was occupied with the 
other. Furthermore, the direct timeline analysis of overlapping events on the two workstations, 
presented in Figures R9-R11, indicates that the total time to do certain pairs of tasks appearing in 
the two workstations concurrently exceeds the sum of the time that would be predicted for each 
task alone. The fact that the actual time exceeds the time predicted even by a single channel 
model, suggests that there is an added switching penalty for going between the two tasks (Miller, 
2002 ). 
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However it is also important to note that these strict single channel theory predictions 
were mitigated in two respects. First, the automation condition, and in particular, the auditory 
condition, reduced the magnitude of single channel behavior. Indeed, in the auditory condition, 
the Figures R9-R11 actually indicate positive evidence for concurrent processing, consistent with 
a multiple resource account. In the automation condition, this reduction in the apparent 
magnitude of single channel behavior is consistent with the findings of Liao and Moray (1993), 
that when the number of tasks is reduced, more evidence arises for parallel processing. 

The second mitigation of strict single channel processing predictions was evident from 
Figure R12, in which it can be seen that the predictions of a linearly increasing delay in 
processing of the second arriving task as its arrival time is earlier, are clearly not confirmed. In 
contrast to the findings of Kerstholt et al. (1996), or Moray and Rotenberg (1986), operators in 
the current domain were apparently not postponing action on the second arriving task until the 
first was completed, a behavior that, if observed, would have reflected strict single channel 
processing. As described above, there is evidence that pilots may be sampling each display at 
approximately 3-second intervals (on the average). Presumably, faced with the concurrent 
processing demands of “examination tasks” on each workstation, pilots alternated processing of 
each in such a way that both tasks bore the brunt of their concurrency. Indeed, this added delay 
with reduced arrival interval might be shown in the first arriving task, although such data are not 
yet analyzed. It is also consistent with the switching cost data in Figures R9-R11 that showed 
pilots switching between tasks with some frequency, given that the cost of a switch in task 
activity can be substantial (Miller, 2002). 

Some aspects of single resource theory were also in evidence. We have already noted the 
consistency of the current data with the “effort-mobilizes more resources” conception put forth 
by Kaheman (1973), in noting the elimination of single channel cognitive tunneling for the 
auditory condition. Here the contrast with the previous experiment is important. The added 
incentives for excellent performance in the current experiment, absent in the previous, 
presumably caused pilots to invest more effort into the task, thereby mobilizing more resources, 
and mitigating dual task decrements in the auditory condition where there was no peripheral 
bottleneck We also find that, with some exceptions, decrements in dual task conditions are not 
substantially greater than in single task conditions. This can be ascertained by comparing the 
decrements in perfonnance from the baseline to either of the offloads, or from the “easier” SF 
type (SF b ) to the more challenging type (SF D and SF E ), between single and dual task conditions. 
Single channel theory would typically predict a doubling of those decrements (“massive 
interference”), something not found here. The reduced, but still present decrements, are 
consistent with resource theory. 

The extent to which single resource theory can adequately account for the data can be 
best evaluated by correlating the decrement values with the sum of task demand values, as shown 
in Table 2. To the extent that such correlations are high, it suggests that most variance in 
workload (here related to task overload performance costs) can be accounted for simply by 
variance in how difficult the tasks are rated to be. As our analysis showed, correlations of model 
fits for some of the performance measures, TOO detection rate and # of CT repeats were quite 
high, with the single resource demand model accounting for over 85% of the variance in 
performance across the six conditions. At the same time, the single resource model fit was less 
impressive in accounting for variance in SF monitoring performance. This is not surprising, 
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given the strong roll of multiple resources in influencing performance on the SF task, as 
discussed above. We should note however that there are other ways of making single resource 
model predictions. On the one hand, we could have used pilots rating of task difficulty as our 
inputs to the demand vector, rather than our own analyst ratings. On the other hand, we could 
have made more refined predictions by predicting different epochs of perfonnance of a given 
task, whose resource demands might change (e.g., making separate predictions for different 
system failure types). This remains to be done. 

In conclusion, it does appear that different aspects of all three models, account for some 
parts of the data. Single channel theory accounts nicely for the task switching effects observed. 
However single channel theory cannot really handle aspects of differential task difficulty, except 
when those differences translate directly into task time. However in the case of tasks like SF 
monitoring or tracking monitoring, these really do not lend them selves to time measures, since 
they are continuous ongoing tasks. Hence there is value added in considering task demands; and 
as we saw, such considerations were very useful in accounting for variance in some, but not all 
tasks. Finally, it appears that multiple resource theory is responsible for accounting for some 
substantial aspects of time sharing performance that were not accounted for by models based on 
the other two theories, particularly the difference between auditory and visual delivery. An 
effective computational model therefore needs to incorporate all three characteristics: resource 
demand (SRT) and resource structure (MRT), along with some single channel switching 
assumptions, that characterize how task are prioritized in sequence, when demand or structure 
are such that concurrent task performance is impossible. 

With regard to the fundamental issue addressed by this study, is single operator control of 
two UAV's feasible? The answer is only a very qualified "yes". The basic mission requirements 
were met with two, as well as one UAV. However, even with the various offloads, dual UAV 
requirements sacrificed some aspects of vital secondary task perfonnance and in particular, 
monitoring for targets of opportunity. And these findings were observed in "best case" 
circumstances in which automation was essentially perfect. Following the review of Wickens 
and Xu (2002), we anticipate substantial problems in dual UAV control with imperfect 
automation. 
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